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This  research  is  designed  to  identify  Massachusetts  school  districts  whose 
student  test  scores  exceed  the  scores  predicted  by  their  demographic 
characteristics.   The  work  is  not  intended  to  rank  districts'  performances  but 
rather  to  highlight  the  efforts  of  districts  whose  students  are  exceeding  what 
they  would  be  expected  to  achieve  on  statewide  standardized  tests.   The  goal  is 
to  enable  other  districts  to  study  and  learn  from  the  efforts  of  systems 
identified  as  effective  in  this  work. 
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Education,  then,  beyond  all  other  devices  of  human  origin, 
is  the  great  equalizer  of  the  conditions  of  men .. . 

-  Horace  Mann 


"Good  education  relieves  pressure  on  other  socl\l  support 
systems;  a  weak  education  system  stresses  all  other  systems  and 
services,  including  industry.  in  short,  public  education  is 
central  to  america' s  quality  of  life  and  economic  future. " 

-  Jade  Rennie,  prime  mover  behind 
the  Education  Reform  Act  of  1993 


OVERVIEW:  SCHOOLDISTRICT  EFFECTIVENESS  ON  THE  2000  MCAS 


The  first  analysis  of  school  district  effectiveness  came  out  in  February  of  1999 
and  evaluated  the  1998  MCAS  in  terms  of  district  demography.  The  second 
report  was  released  in  February  2000  and  considered  the  1999  MCAS.  The 
central  tool  of  these  analyses  is  the  Effectiveness  Index  methodology  that 
examines  the  relationship  between  selected  commuiuty  demographic 
characteristics  and  educational  outcomes.  These  characteristics  include: 
average  education  level,  average  income,  poverty  rate,  single-parent  status, 
language  spoken,  and  percentage  of  school-age  poptQation  enrolled  in  private 
sdiools.  These  variables  were  chosen  because  they  correlate  with 
aduevement  and  because  the  education  literature  identifies  them  as 
connected  to  academic  performance.  (See  Appendix  F  for  information  on  the 
Effectiveness  Index.) 

Researchers  ranging  from  James  Coleman  in  the  1960s  to  James  Comer  in  the 
1990s  have  demonstrated  that  commimity  demographics  play  a  major  role  in 
how  well  children  do  in  school  The  Effectiveness  Index  model  provides  a 
means  of  isolating  the  role  played  by  community  characteristics  concerning 
student  performance  on  statewide  educational  assessments.  With  a 
commtmity's  achievement  context  factored  into  its  test  results,  it  is  possible  to 
know  how  much  value  school  systems  add  to  demographic  expectations.  In 
the  absence  of  a  methodology  to  control  for  the  demographic  diversity  of 
Massachusetts,  listing  MCAS  scores  primarily  demonstrates  the  relative 
advantage  or  disadvantage  that  commimity  characteristics  bring  to  students. 
Any  raw  ranking  order  of  MCAS  scores  reflects  district  demography  much 
more  than  it  represents  anything  else.  A  sorting  of  MCAS  results  would  tell 
us  more  about  local  real  estate  values  or  the  percentage  of  SUV  ownership  in 
a  commimity  than  it  would  about  school  quality. 

The  Effectiveness  Index  identifies  school  districts  that  add  value  to  the 
learning  readiness  of  their  students  as  indicated  by  higher-than- 
demographically-predicted  test  scores.  Identifying  such  systems  is  a  first  step 
to  determining  if  they  are  indeed  providing  more  effective  educational 
services  to  their  students.  Identifying  best  practices  in  effective  systems  that 
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are  demographically  similar  to  less  effective  systems  may  help  those  systems 
improve  their  school  services. 

There  are  several  details  of  note  about  this  report 

•  Small  districts  (those  with  fewer  than  45-50  students  taking  MCAS 
exams  in  one  grade)  are  not  included  in  this  report  This  is  not  a 
reflection  of  a  bias  against  small  systems.  Rather,  small  sample  size 
can  add  significant  error  to  any  statistical  analysis. 

•  Grade  8  History  is  not  evaluated  in  the  study. 

•  This  third  edition  covers  98%  of  the  state's  ptiblic  school  population 
in  Grades  4  and  8  and  88%  of  Grade  10  students.  (Regional 
vocational-technical  schools,  which  are  not  included  in  this  report, 
educate  10%  of  the  Grade  10  students.) 

Massachusetts  students  attend  school  in  district  systems  and  in  regional 
systems.  Most  students  attend  district  systems  while  regionals  educate  9%  of 
the  Grade  4  students  and  15%  of  the  Grade  8  and  Grade  10  students  in 
Massachusetts.  District  school  districts  (e.  g.,  Boston,  Wobum,  South  Hadley) 
serve  students  from  only  one  commimity.  Regional  systems  serve  students 
from  several  communities.  For  example,  Nashoba  Regional  educates  children 
from  the  towns  of  Bolton,  Lancaster  and  Stow.  In  the  case  of  evaluating  the 
effectiveness  of  regional  school  systems,  community  demographics  have  been 
factored  to  reflect  the  regional  school  district  characteristics. 
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OBSERVATIONS 


•  As  was  the  case  last  year,  there  are  many  repeat  performers  in  the 
Effective  Districts  list.  Generally,  about  half  of  the  sixty  systems  identified  in 
each  grade  and  subject  area  (Grades  4, 8,  and  10  English  Language  Arts,  Math, 
Science  and  Technology)  in  this  report  as  adding  value  to  the  demographic 
characteristics  of  their  students  are  repeaters  from  last  year.  This  is  not 
stirprising;  a  system  that  had  oiganized  itself  to  enhance  student  achievement 
in  1999  is  likely  to  have  kept  that  up  in  2000.  (See  Appendix  B  for  a  listing  of 
systems  that  were  identified  as  over-performing  in  1999  and  2000.) 

•  Districts  that  over-perform  their  demography  tend  to  be  middle-class 
or  demographically  advantaged  commimities.  Generally,  upper-demography 
commtmities  are  about  two  times  more  likely  to  over-perform  than  are 
communities  of  lower  demography.  I  define  upper  demography 
commimities  as  being  in  the  top  25%  of  Massachusetts  communities  based  on 
my  demographic  methodology;  I  define  lower  demography  commimities  as 
those  in  the  bottom  25%  of  the  state  in  terms  of  such  demography.  (See  page 
22  for  more  information  on  community  demography.) 

The  relative  lack  of  capacity  of  lower  demography  conununities  to  exceed 
their  demographic  characteristics  on  standards-based  assessments  is  a 
persistent  point  of  concern.  Students  from  middle  and  upper-middle  dass 
communities  generally  do  well  on  any  assessments,  including  MCAS.  For 
such  districts,  demography  is  not  a  major  barrier  to  success.  Indeed,  for  many 
districts,  demography  is  a  guarantor  of  success.  Reforming  various  elements 
of  their  educational  offerings  will  probably  be  enough  to  bring  virtually  all 
students  up  to  state  standards. 

For  demographically  challenged  districts,  demography  is  a  consistent  drain  on 
achievement  Our  cities  and  distressed  towns  need  to  over-perform  their 
demography  for  their  students  to  develop  the  basic  skills  needed  to  live  and 
woric  in  Massachusetts.  Based  on  three  years  of  MCAS,  most  of  the  students 
in  middle  and  upper-demography  districts  perform  well  enough  now  to  pass 
(achieve  one  point  above  Fail)  the  MCAS  graduation  requirements.  Districts 
that  are  disadvantaged  have  much  more  work  to  do  to  overcome  their 


demography  and  lift  more  of  their  students  into  success  in  school.  After  three 
MCAS  administrations^  it  is  dear  that  disadvantaged  districts  are  having  a 
hard  time  implementing  the  dramatic  school  reforms  needed  to  help  their 
teachers  and  students  outperform  their  commimity  characteristics. 

•  Some  districts  have  compiled  a  solid  three-year  record  of  over- 
performing  their  demography.  It  might  be  helpful  to  study  those  districts 
that  have  consistently  over-performed  in  specific  subject  areas.  The 
following  lists  identify  the  districts  in  the  state  that  had  the  greatest  over- 
performance  in  ELA  and  Math,  combining  results  from  the  1998,  1999,  and 
2000  MCAS. 

•  In  Grade  4  ELA,  Wobum  and  Eastham  scored  16  scaled  score 
points  above  each  district's  predicted  performance  level 

•  In  Grade  4  Math,  Orleans  scored  29  points  above  predicted 
score,  with  Harvard  scoring  25  points  above  its  predicted 
result. 

•  In  Grade  8  ELA,  Stoneham  scored  17  points  above  predicted 
score  with  Harvard  scoring  14  points  over  prediction. 

•  In  Grade  8  Math,  Harvard  scored  31  points  and  Ipswich  scored 
24  points  over  predicted  score. 

•  In  Grade  10  ELA,  Stoneham's  actual  score  was  24  points  over 
predicted  score,  and  Hatfield's  students  scored  23  above 
predicted  performance. 

•  In  Grade  10  Math,  Harvard  once  again  topped  the  over- 
scorers  with  42  points  over  predicted  score,  and  Lenox  scored 
34  over  predicted  score. 

(Appendix  C  lists  districts  that  have  demonstrated  the  most  over- 
performance  over  the  three  MCAS  administrations.) 

•  Many  districts  that  over-performed  their  demography  did  so  without 
any  apparent  benefit  from  high  per-pupil  school  spending  or  high  levels  of 
new  state  education  reform  aid.  Generally,  over-performers  spent  at  or  below 
state  average  and  were  not  the  recipients  of  generous  amounts  of  Chapter  70 
education  reform  aid.  Braintree  and  Stoneham,  two  of  the  most  effective 
systems  in  this  analysis,  receive  $336  and  $319  respectively  per-pupil  under 
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education  reform.  The  average  amount  of  education  reform  aid  per-pupil  in 
Massachusetts  imder  education  reform  is  $1,263. 

It  is  importcmt  to  note  that  this  does  not  mean  that  reform  money  does  not 
matter.  This  observation  illustrates  two  points: 

•  The  education  reform  funding  formula  is  designed  to  equalize  funding.  A 
district  spending  at  or  near  its  formula-determined  per-pupil  amount 
when  the  law  was  passed  would  not  receive  substantial  additional 
funding.  Braintree  and  Stoneham  both  spent  above  state  average  in  1994 
when  the  reform  law  became  functionaL  These  districts  did  not  need  to 
have  tiieir  spending  supplemented  by  significant  state  aid. 

•  It  is  easier  to  implement  reform  in  school  districts  that  do  not  face  major 
demographic  challenges  as  they  work  to  improve  pedagogy.  Money 
matters  in  demographically  challenged  districts,  but  until  their 
educational  delivery  systems  are  adapted  to  meet  student  needs  more 
effectively,  more  money  will  not  guarantee  success. 

Under  the  state  education  reform  funding  formula,  demographically 
disadvantaged  systems  generally  receive  much  more  additional  school 
funding  from  the  state  than  do  middle-class  or  advantaged  systems.  For 
example,  under  education  reform,  Lawrence,  the  most  demographically 
disadvantaged  system  in  the  state,  receives  $3,643  per  student  per  year  in 
reform  aid.  Many  upper-demography  commtmities  receive  $300  or  less  per 
student  per  year  in  such  aid. 

Even  with  additional  state  ftmding,  demographically-challenged 
communities  like  Lawrence  face  formidable  obstacles  in  reforming  their 
schools  to  the  extent  necessary  to  educate  their  students  to  meet  high 
standards.  (See  Footnote  11  for  information  about  calculating  the  amount  of 
new  education  reform  aid.  Appendix  C,  which  lists  those  districts  that  have 
had  the  greatest  over-performance  on  three  years  of  MCAS  assessments,  also 
lists  the  amoimt  of  new  money  districts  receive  under  education  reform.) 
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The  good  news  here  is  that  some  of  the  districts  that  substantially  over- 
perform  their  demography  did  receive  substantially  more  state  aid  as  a  result 
of  education  reform.  These  districts  tend  to  be  part  of  Middle  Massachusetts, 
the  140  or  so  commtmities  that  are  neither  demographically  advantaged  or 
disadvantaged.  For  example,  Carver,  Leicester,  and  Uxbridge,  all  Middle 
Massachusetts  commtmities,  solidly  over-performed  their  demography  on 
the  2000  MCAS  in  terms  of  total  score  (English  Language  Arts,  Math,  Science) 
and  also  received  substantial  amotmts  of  new  state  funding  under  education 
reform. 

This  may  suggest  that,  in  the  case  of  reform-sawy  middle-class  communities 
that  were  tmderfunded  before  the  Education  Reform  Act  of  1993,  additional 
fimding  has  been  spent  in  ways  that  have  boosted  student  achievement. 
Again,  improving  the  way  educational  services  are  delivered  is  easier  in 
districts  that  do  not  face  formidable  demographic  challenges  in  terms  of  the 
learning  readiness  of  their  students. 

•  One  sobering  observation  from  the  1998  and  1999  reports  is  still  true. 
So  far,  after  seven  years  of  increased  reform  funding  to  many  systems,  there  is 
little  evidence  that  many  of  our  schools  have  changed  in  any  fimdamental 
ways.  MCAS  scores  were  relatively  flat  from  1998  to  1999  to  2000.'  There  is 
certainly  good  news  in  the  scores  of  some  districts,  but  in  the  main,  the  pace 
of  progress  has  not  matched  the  hopes  and  expectations  of  many  citizens. 

•  Demography  matters  less.  When  this  study  was  first  done  on  the 
1998  MCAS,  commimity  demography  accounted  for  86%  of  the  variation  in 
total  MCAS  test  scores  across  the  Commonwealth.  In  the  2000  MCAS  overall 
total  score,  demography  accoimts  for  slightly  less  (83%).  Here,  however,  is 
one  situation  where  focusing  on  the  overall  picture  may  mask  important 
detail.  Demography  matters  most  when  looking  at  total  MCAS  scores; 


^  There  was  a  6  point  average  scaled  score  increase  in  Grade  10  Math  and  a  4  point  increase  in 
average  scaled  score  for  Grade  8  and  Grade  10  Science  and  Technology  between  1999  and  2000. 
Other  Grade/ subjects  displayed  1  or  2  point  increases.  See  Spring  2000  MCAS  Tests:  A  Report  of 
State  Results  (November  2000],  available  on  the  Massachusetts  Department  of  Education's  web 
site  -  www.  doejnass.edu/  mcas/ . 


demography  matters  relatively  less  when  considering  individual  grade  and 
subject  areas. 

Of  particular  note  is  the  fact  that  for  ELA4  (Grade  4  reading),  the  amount  of 
variation  in  test  score  accotmted  for  by  demography  dropped  from  72%  in 
1999  to  64%  in  2000.  Similarly,  the  amount  of  score  variation  accoimted  for  by 
demography  dropped  from  68%  to  60%  for  ELAIO,  Grade  10  reading. 

This  is  encouraging.  As  the  impact  of  demography  on  reading  outcomes 
shrinks,  the  impact  of  what  happens  in  the  classroom  grows.  The  reduced 
role  of  demographics  in  accounting  for  reading  scores  is  evidence  that 
education  reform  has  made  the  school  a  more  powerful  force  in  shaping 
reading  outcomes  over  the  past  three  years.  This  is  good  news  in  that 
developing  reading  skills  is  fundamental  to  any  successful  school  reform 
effort  Results  on  the  2000  MCAS  indicate  that,  while  still  the  dominant 
factor  in  achievement,  demography  matters  less  than  it  used  to  for  some 
subjects  and  grades. 
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THEMCAS 


Testing  plays  an  important  role  in  most  of  the  contemporary  school  reform 
efforts  in  the  United  States.  The  Massachusetts  education  reform  effort  is  no 
exception.  Its  testing  vehide  is  the  Massachusetts  Comprehensive 
Assessment  System  or,  as  it  is  commonly  known,  the  MCAS. 

MCAS  tests  have  been  part  of  the  education  landscape  in  Massachusetts  since 
Spring  of  1998.  The  assessments  were  developed  imder  the  1993  Education 
Reform  Act  that  substantially  increased  state  aid  for  education.  The 
legislation  created  MCAS  to  give  educators  information  about  how  to 
improve  teaching  and  learning  and  to  determine  how  much  improvement 
was  actually  taking  place  in  the  schools  as  a  result  of  the  new  money  and  the 
reform  package.  The  tests  are  designed  to  assess  student  performance  across 
the  state. 

The  MCAS  is  a  battery  of  tests  that  have  been  given  each  year  to  students  in 
Grades  4,  8,  and  10  in  each  school  district  in  1998, 1999,  and  2000.  The  MCAS 
is  aligned  with  a  series  of  curriculimi  frameworks  developed  by  the  state 
Department  of  Education.  MCAS  covers  such  academic  subjects  as  math, 
science,  history,  and  literacy  skills.  The  test  scores  are  broken  down  by 
individual  student  school,  and  district.  The  scores  for  individual  students  are 
available  to  their  parents,  teachers,  principals,  and  superintendents.  The 
scores  for  entire  sdiools  and  districts  are  available  to  the  general  public. 
Students  are  expected  to  pass  (achieve  a  score  of  one  point  above  Fail)  the 
Math  and  English  Lang\iage  Arts  10th  Grade  MCAS  test  as  a  high  school 
graduation  requirement. 

The  chief  objective  of  the  state's  education  reform  initiative  is  to  enable 
public  school  students  to  achieve  a  certain  level  of  knowledge  and  skill.  The 
Massachusetts  Department  of  Education  has  established  this  level  by  setting 
out  what  students  are  expected  to  learn  in  each  basic  subject  School  districts 
are  supposed  to  see  to  it  that  their  students  learn  what  they  are  expected  to 
learn. 
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MCAS  has  three  broad  purposes;  Performance  Assessment;  Diagnostics;  and 
Accoimtability. 

•  The  Performance  aspect  of  the  MCAS  -  the  scores  -  has  been  the  most 
publicized  part  of  education  reform.  Newspapers  and  electronic  media 
regularly  report  scores  of  districts. 

•  The  Diagnostics  built  into  MCAS  give  educators  the  capacity  to  identify 
specific  learning  weaknesses  in  individual  students  as  well  as  target 
problems  in  curriculum  and  teaching  that  may  impede  achievement. 
School  systems  should  be  able  to  improve  teaching  and  learning  through 
utilization  of  these  diagnostics. 

•  Finally,  MCAS  gives  citizens  an  Accountability  tool  that  measxires  how 
well  schools  are  doing  in  moving  their  students  towards  high 
achievement.  With  MCAS  data  over  time  as  a  gviide,  we  should  be  able  to 
increase  the  accountability  of  individuals,  schools,  and  systems. 

With  the  MCAS,  the  state  has,  for  the  first  time  in  its  history,  an  evaluation 
mechanism  that  measures  how  much  progress  students  are  making  towards 
meeting  well-defined  goals.  At  the  same  time,  individual  school  districts  are 
urged  to  complement  the  MCAS  by  developing  their  own  parallel  methods  of 
assessing  how  their  students  are  doing.  The  Massachusetts  education  reform 
effort  uses  assessment  as  a  way  to  help  all  students  move  towards  a  high  level 
of  academic  achievement. 

Just  as  this  education  improvement  effort  views  higher  student  achievement 
as  its  end,  it  views  the  improvement  of  the  public  schools  as  its  chief  means 
to  achieve  this  end.  What  happens  in  school  is  by  no  means  the  only  or  even 
the  leading  influence  on  how  pupils  currently  perform  on  standardized 
academic  tests.  However,  what  happens  in  school  obviously  is  the  only 
means  that  is  currently  within  the  control  of  the  schools  and  school  systems 
themselves.  School  change  is  the  only  means  of  reform  that  is  at  the  disposal 
of  the  education  improvement  effort  as  it  now  exists. 
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MCAS:  Fair  of  Foul? 


Some  have  criticized  MCAS  for  various  reasons:  the  tests  stifle  educational 
creativity;  standardized  tests  are  inherentiy  flawed;  one  test  should  not  be  the 
determinant  of  being  given  a  high  school  diploma.  There  certainly  is  much 
room  to  debate  the  specifics  of  any  major  educational  policy^  but  it  may  be 
unwise  to  blame  the  test  for  its  findings.  When  one  is  diagnosed  with  high 
blood  pressure  and  high  cholesterol,  one  can  always  ignore  the  diagnosis  and 
criticize  the  doctor.  That  type  of  behavior  only  ensures  substantial  problems 
down  the  road. 

Much  of  the  anti-MCAS  activity  focuses  on  the  eventual  high-stakes  nature 
of  MCAS  which  calls  for  students  to  demonstrate  a  basic  mastery  of  English 
Language  Arts  and  Math  as  a  condition  of  receiving  a  high  school  diploma. 
Paul  Reville,  an  architect  of  contemporary  education  reform,  has  noted  that 
the  focus  on  the  score  aspect  of  MCAS  has  clouded  the  fact  that  MCAS  is 
designed  "to  be  a  tool  to  help  educators  establish  a  data-driven  system  of 
education."^  Because  of  MCAS,  educators  have  solid  information  arotmd 
which  to  organize  their  teaching. 

The  bulk  of  the  criticism  has  come  from  advantaged  communities,  places 
where  the  schools  are  thought  to  function  reasonably  well.  As  Thomas 
Fortmann,  a  citizen  with  successful  careers  in  high  technology  and  urban 
education,  points  out 

MCAS  is  not  a  major  problem  in  wealthy  districts  where 
functional  schools,  parental  support,  and  high  expectations 
ensure  that  most  graduates  are  both  Uterate  and  numerate. 
From  a  comfortable  subiuban  vantage  point,  if  s  easy  to  criticize 
the  tests  and  denounce  the  graduation  requirement  If  s  easy, 
but  if  s  wrong  -  because  it  hurts  kids  in  less  fortunate  districts.^ 


^  See  Paul  Reville,  'MCAS  isn't  failing  students,"  Boston  Herald,  December  26, 2000,  p.  25. 
'  Thomas  Foctmann,  'Students  Need  MCAS  Tests,"  Boston  Globe,  November  18, 2000,  p.  A 11.. 


By  any  standard  -  SAT  scores,  district-wide  standardized  tests,  employment 
assessments,  commimity  college  student  assessments  and  MCAS  -  there  is  a 
vast  range  of  performance  demonstrated  by  students  in  Massachusetts. 
Literally,  a  town  where  the  overwhelming  majority  of  students  pass  MCAS 
can  be  situated  next  to  a  dty  where  most  students  do  not  pass.  Andover  and 
Lawrence,  Brookline  and  Boston,  Shrewsbury  and  Worcester  all  provide 
graphic  evidence  of  such  achievement  disparity.  On  the  2000  MCAS: 

-  In  Andover,  87%  passed  the  Grade  10  ELA;  81%  passed  the 
Grade  10  Math.  In  Lawrence,  33%  passed  the  ELA  and  23% 
passed  Math. 

-  In  Brookline,  81%  passed  the  Grade  10  ELA;  78%  passed  the 
Math.  In  Boston,  44%  passed  ELA  and  34%  passed  Math. 

-  In  Shrewsbury,  92%  passed  the  Grade  10  ELA;  82%  passed  the 
Grade  10  Math.  In  Worcester,  49%  passed  ELA  and  38% 
passed  Math. 

Rather  that  criticizing  the  messenger,  it  would  be  better  to  use  the  MCAS  as  a 
catalyst  for  the  radical  changes  needed  in  our  urban  systems  to  make  them 
able  to  educate  their  yoimg  people.  While  this  may  be  inconvenient  or 
politically  offensive  to  some  people  who  live  in  our  more  advantaged 
communities,  using  MCAS  to  prod  effective,  reflective  change  would  make 
for  a  much  more  equitable  Commonwealth  for  the  new  centiuy. 

Without  substantial  changes  in  urban  and  other  disadvantaged  schools,  a 
student's  educational  success  will  continue  to  be  a  function  of  zip  code.  The 
goal  of  education  reform  should  be  to  change  the  finding  of  the  Willis- 
Harrington  Commission  of  1965  that  found  that,  concerning  educational 
quality,  "It  matters  vitally  to  every  individual  where  the  accident  of  birth  and 
home  locates  him."  Thirty-five  years  after  the  most  comprehensive 
education  study  in  state  history  spoke  of  the  impact  of  demography  and 
geography  on  achievement,  we  still  face  the  challenge  of  neutralizing  the 
impact  of  demographics  on  educational  outcomes.* 


*  See  Massadtusetts  General  Court,  Special  Commission  to  Itwestigate  and  Study  Educational 
Facilities  in  the  Commontpealth  of  Massadtusetts  (Bostoiv  1965)  p.  82.  Also,  Robert  D.  Gaude 


IMPROVING  OUR  SCHOOLS 


The  more  test  scores  can  be  used  to  inform  decisions  about  how  to  alter  what 
happens  in  school,  the  better  the  chances  to  make  the  schools  more  effective 
in  helping  their  students  learn  more  and  improve  their  performance  on 
standardized  academic  tests  like  the  MCAS.  Properly  \ised,  the  resiilts  can 
pinpoint  which  approaches  to  teaching  and  learning  are  working  and  which 
are  not  The  essence  of  education  reform  in  Massachusetts  can  be  stunmed 
up  in  a  few  words:  Better  student  performance  through  more  effective 
schools. 

However,  for  the  MCAS  to  fulfill  its  intended  role  in  the  current  education 
reform  effort,  there  at  least  two  important  conditions  that  have  to  be  met. 

FIRST,  the  tests,  and  other  assessments,  must  be  fair  and  accurate. 
They  must  measure  what  children  have  learned,  rather  than  just  their 
social  or  economic  backgroimd.  They  must  not  be  biased  for,  or  against, 
any  group  of  students. 

SECOND,  assessments  mtist  be  used  to  make  the  public  schools  more 
effective.  Thus,  the  scores  should  drive  an  ongoing  analysis  of  what 
makes  the  school  experience  effective.  They  must  provide  teachers 
with  a  critical  piece  of  information  about  the  potential  learning 
problems  and  possibilities  of  individual  students.  Ultimately,  the 
information  should  be  used  as  a  basis  for  helping  all  students  to  do 
better. 

To  meet  the  second  condition,  we  must  be  able  to  use  the  MCAS  scores  as  one 
tool  to  discern  the  effectiveness  of  our  schools.  We  must  be  able  to  establish 
how  effective  they  are  today,  and  to  track  the  rise  or  fall  of  their  effectiveness 
in  the  future.  Thus,  finding  ways  to  measure  school  effectiveness  is  essential 
to  education  reform. 


The  Willis  Harrington  Commission:  The  Politics  of  Education  Reform,"  New  England  Journal 
of  Public  Policy  3(Summer/Fall  1987):  66-77. 


Measuring  Educational  Effectiveness 


Student  academic  performance,  including  how  students  do  on  MCAS  tests,  is 
influenced  by  two  broad  sets  of  factors:  school  factors  and  non-school  factors. 
The  first  entail  what  happens  in  school  and  thus  what  is  within  the  control 
of  the  school  district  itself.  The  second  entails  conditions  outside  the  schools, 
such  as  the  demographic  profile  of  the  students  and  the  community.  As  we 
look  at  a  given  district's  average  score  on  an  MCAS  test^  we  have  to  be  able  to 
discern  how  much  of  the  score  is  tied  to  school  factors,  and  how  much  of  the 
score  is  explained  by  non-school  factors. 

How  well  does  the  school  design  and  the  curricultun  promote  learning  for 
all?  Are  teachers  top-notch  professionals  who  have  both  the  skills  and 
commitment  to  teach  all  students?  Are  professional  development  activities 
rigorously  aligned  with  efforts  to  increase  student  achievement?  Is  there 
strong,  soUd  leadership  in  the  school?  Are  there  high  expectations  for  aU? 
Are  parents  full  partners  in  their  children's  education?  Are  there  adequate 
resources  to  do  the  job?  These  are  all  questions  about  school  factors.^ 

In  the  research  reported  in  this  paper,  non-school  factors  consist  largely  of  the 
overlapping  demographic  conditions  of  family  life  and  commimity  life.  This 
study  uses  six  such  conditions  in  a  given  school  district:  its  median  level  of 
educational  attainment,  its  median  income  level,  its  percentage  of 


'Per  pupil  expenditure  [PPE]  is  a  basic  school  factor,  but  our  measures  of  it  are  not  always 
complete.  For  example,  as  reported  on  the  Boston  Public  Schools  web  site 
(ht^.//www.bostonJ(12.nu.us/bps/  bpsglance.asp#money),  Boston's  per-pupil  expenditure  is 
$9,025  The  web  site  also  informs  us  that  Boston  raised  $%,617,594  in  FY99  through  External 
Funds.  Thus,  an  accurate  accounting  of  the  educational  resources  Boston  has  would  have  to 
indude  these  external  funds.  Based  on  a  63,300-student  population,  external  funding  increases 
the  per-pupil  expenditure  by  $1,526,  thus  giving  Boston  a  true  per-pupil  expenditure  of  $10,551. 
Cheddng  out  the  Massadiusetts  Department  of  Education  web  site  yields  different  information. 
The  DC^  Sdiool  Profiles  sets  out  a  figure  of  $8,118  as  the  day  program  per-pmpil  expenditure. 
Part  of  the  discrepancy  ($8,118  reported  by  DOE;  $9,025  reported  by  the  Boston  Public  Schook) 
may  be  because  the  Boston  Public  School  figure  reflects  the  current  fiscal  year  whereas  the  DOE 
figure  reflects  the  PPE  of  a  previous  fiscal  year.  In  any  event,  the  DOE  figure  does  not  indude 
the  external  funds  that  comprise  a  significant  portion  of  the  money  Boston  spends  on  public 
education.  One  must  be  careful  in  determining  what  our  systems  are  actually  spending  on  public 
education. 


hotiseholds  above  the  poverty  line,  its  percentage  of  single-parent  families,  its 
percentage  of  non-English-speaking  households,  and  its  level  of  private 
school  enrollment.  Statistical  analysis  shows  that  these  factors  form  much  of 
the  non-school  influence  on  how  the  state's  students  do  on  such  standardized 
tests  as  the  MCAS.* 

As  we  all  know,  students  in  advantaged  districts  tend  to  get  higher 
standardized  test  scores  than  students  in  disadvantaged  districts.  Thus,  if  a 
district's  students  get  a  high  average  score  on  an  MCAS  or  other  standardized 
test,  the  test  score  by  itself  does  not  tell  us  how  much  of  the  score  is  explained 
by  sdiool  factors  and  how  much  is  explained  by  non-school  factors.  A  high 
score  might  be  tied  more  to  advantaged  demography  than  to  what  actually 
happens  in  the  district's  schools.  The  score  by  itself  is  not  a  sound  guide  to 
how  effective  is  the  school  district. 

We  cannot  begin  to  zero  in  on  just  how  effective  the  school  district  itself  is 
tmless  we  can  distinguish  between  the  respective  influences  of  the  two  types 
of  factors.  Only  then  can  we  discern  how  effectively  the  district  itself 
performs,  and  how  much  it  contributes  to  its  students'  average  performance 
on  the  MCAS. 

Identifying  Over-performing  Systems 

Identifying  systems  that  over-perform  their  demography  is  important  in  that 
such  systems  may  have  valuable  lessons  to  offer  similar  systems  in  their 
efforts  to  boost  student  achievement  In  fact,  the  state  Department  of 
Education  and  Masslnsight  Education,  a  well-respected  non-profit  education 


Other  feunily  and  oommunity  conditions  are  crudal  to  student  success,  but  are  hard  to  observe 
and  measure.  One  would  have  to  monitor  many  families  and  communities  dosely  over  time  to 
discern  how  family  and  community  behavior  affect  sdnwl  outcomes.  How  many  books  are  read 
in  the  famify?  How  mudi  time  is  taken  up  by  watching  TV?  How  do  the  community's  adults 
treat  duldren  other  than  their  own?  Does  die  omununity  mentor  its  young  people?  It  is  hard  to 
get  reliable  answers  to  sudi  questions.  But  we  do  know  dvat  die  duldren  of  advantaged 
families  and  communities  are  more  likely  on  average  to  have  resources  and  support,  and 
duldren  of  less  advantaged  situations  are  less  likely  to  have  theuL  So  we  use  gross  measures  of 
such  support  as  a  proxy  for  answers  to  the  more  specific  questions  that  are  so  hard  to  pursue. 
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reform  oi^anizatioiv  are  working  together  to  identify  Exemplary  Schools  and 
determine  which  specific  initiatives  have  contributed  to  the  outstanding 
performance  of  a  school  or  district  (See  the  Masslnsight  web  site  for  more 
information  -  www.massinsight.com.) 

It  is  especially  important  to  identify  over-performing  demographically- 
challenged  systems.  After  three  years  of  MCAS,  we  know  that  many  of  our 
less  advantaged  districts  still  have  far  to  go  to  meet  new  state  standards. 
Helping  these  systems  move  ahead  is  critically  important  to  ultimate  success. 
When  we  identify  low  demography /over-performing  systems,  we  need  to 
study  them  carefuUy  and  learn  the  lessons  they  offer  on  how  to  reach  and 
teach  all  children. 

The  Effectiveness  Index 

The  Effectiveness  Index  (EI)  provides  a  measure  of  the  school  district's 
contribution  to  its  student  performance.  The  Effective  Index  supplies  a  piece 
of  crucial  insight  as  to  which  school  districts  are  more  effective. 

For  a  given  district,  the  Effectiveness  Index  gauges  the  impact  that  school 
factors  have  on  the  average  MCAS  score.  The  greater  the  positive  impact  of 
the  school  factors,  the  higher  the  district's  Effectiveness  Index  will  be. 

The  Index  is  calculated  in  the  following  mannen  For  a  given  district,  the  six 
demographic  factors  are  used  as  the  basis  for  projecting  a  likely  average  score 
on  the  MCAS.  The  demographically-likely  score  is  then  compared  to  the 
average  score  that  the  students  in  the  district  acttially  received.  The 
Effectiveness  Index  is  the  number  that  represents  the  difference  between  the 
likely  score  and  the  actual  score. 

If  the  number  is  negative  -  if  the  actual  score  is  lower  than  the  likely  score  - 
then  this  suggests  that  what  is  happening  in  the  schools  in  the  district  is  not 
enabling  its  students  to  perform  beyond  the  demographic  expectations  for 
them.  If  the  number  is  a  positive  nxmiber  -  if  the  actual  score  is  higher  than 
the  likely  score  -  then  this  suggests  that  what  is  happening  in  the  schools  is 
helping  the  district's  students  to  surpass  the  demographic  expectations  for 
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them.  (For  a  fuller  accoxint  of  the  development  of  the  Effectiveness  Index, 
please  see  Appendix  F.) 

What  the  Effectiveness  Index  Tells  Us:  Statewide  Results 

This  research  applies  the  Effectiveness  Index  methodology  to  the  MCAS 
scores  of  school  districts  in  the  state.  One  of  the  consistent  findings  of  this 
analysis  is  that  demography  explains  most  of  the  variation  in  test  scores  from 
district  to  district  Demography  is  less  important  in  explaining  4th  grade 
reading  which  is  perhaps  the  lindipin  of  successful  education  reform  efforts 
(see  page  8).  That  is  good^  but  demography  still  accounts  for  most  of  how 
much  students  achieve  academically. 

It  is  important  to  understand  that  demography  is  not  necessarily  destiny. 
Where  a  person  is  bom  or  raised  does  not  dictate  how  successful  or 
unsuccessful  that  person  will  be  in  life.  Similarly,  demography  does  not 
determine  educational  achievement  in  all  cases.  There  are  examples  of 
successful  urban  schools  where  disadvantaged  students  do  very  well  by  any 
meastire.  However,  demography  does  create  the  context  in  which  schools 
operate  and  in  which  learning  occurs.  It  is  less  likely  that  students  from  a 
disadvantaged  environment  will  be  as  successful  in  educational  achievement 
as  will  be  students  from  advantaged  situations.  Demography  is  about 
tendency,  not  destiny. 

Results  from  this  year's  research  are  similar  to  results  from  last  year's  work: 
about  83%  of  the  variation  in  total  MCAS  test  scores  (scores  for  all  test-taking 
students  for  the  nine  MCAS  tests  combined)  is  explained  by  demography. 
Last  year  the  figure  was  86%.  In  general^  demography  explains  a  bit  less  of 
individual  grade  level  subject  area  scores.  For  example,  demography  explains 
69%  of  Grade  4  Math  scores,  64%  of  Grade  4  English  Language  Arts  scores,  79% 
of  the  Grade  8  Math  results,  and  68%  of  Grade  10  Math  outcomes.  That 
demography  is  a  powerful  predictor  of  assessment  scores  is  why  Weston  and 
Wayland  have  high  MCAS  scores  and  why  Holyoke  and  Brockton  have  low 
MCAS  scores.  Thus,  though  demography  is  not  destiny,  in  this  case  it  sets  a 
strong  tendency,  a  tendency  that  has  been  confirmed  by  three  applications  of 
MCAS. 

18 


A  simple  way  to  depict  the  respective  contributions  that  demography  and  the 
schools  make  to  the  average  level  of  student  performance  on  the  MCAS  is 
this: 

DEMOGRAPHY  +  school  =  average  score 

Nonetheless,  a  nimiber  of  districts  achieved  test  scores  that  are  significantly 
higher  than  their  demography  predicts.  The  goal  of  education  reform  is  to 
flip  the  relative  weights  of  the  two  elements  in  the  achievement  equation.  If 
education  reform  is  successful,  over  time  we  will  see  that: 

DEMOGRAPHY  +  SCHOOL  =  AVERAGE  SCORE 

becomes  the  norm;  what  happens  in  school  will  be  more  powerful  than 
whatever  learning  readiness  or  lack  thereof  the  children  bring  to  school. 
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EFFECTIVE  AND  NOTEWORTHY  SCHOOLDISTRICTS 


The  Effectiveness  Index  lets  us  identify  two  types  of  school  districts  that  are 
interesting  in  terms  of  education  reform:  Effective  Districts  and  Noteworthy 
Districts. 

An  EFFECTIVE  district  meets  two  specifications: 

•  Its  Effectiveness  Index  is  a  positive  ntimber  -  that  is,  its  actual 
score  on  the  test  is  substantially  higher  than  its  demographically  likely 
score. 

•  Its  actual  score  is  equal  to  or  higher  than  the  average  MCAS 
score  for  the  state  as  a  whole. 

Thus,  a  district  that  meets  both  of  these  specifications  invites  further  scrutiny 
to  determine  whether  its  practices  provide  a  worthwhile  model  for  other 
districts.  Not  all  districts  that  meet  the  two  effectiveness  specifications  will 
prove  to  have  lessons  to  teach  other  systems. 

Braintree,  Stoneham,  Norwood,  and  Millbury  are  examples  of  effective 
school  districts.  Actual  total  MCAS  scores  in  those  districts  are  substantially 
higher  than  their  demographically  likely  scores,  Braintree's  demographic 
rank  places  it  in  the  middle  (#91)  of  the  202  systems  whose  students  took  all 
three  grade  level  MCAS  tests,  but  its  total  MCAS  score  places  in  #3Z 
Stoneham's  total  MCAS  scores  are  48  ranks  higher  than  its  demography; 
Norwood's  scores  are  24  ranks  higher;  and  Millbury's  scores  are  37  ranks 
above  its  demographic  position.  Additionally,  each  of  these  district's  total 
2000  MCAS  score  is  higher  than  the  statewide  average  score.'' 

Of  special  note  this  year  is  the  performance  of  students  in  Qinton,  Ayer,  and 
Gardner.  These  three  communities  are  demographically  challenged  yet 
scored  above  state  average  while  significantiy  adding  to  the  demographically- 


^  In  tenns  of  the  demograpl^  of  die  202  sdioo!  systems  whose  students  took  all  MCAS  tests 
(exduding  GR  8  Histoty),  Braintree  tanks  #91;  in  terms  of  total  MCAS  performance,  Braintree 
ranks  #32.  Stoneham  ranks  #108  demographically  and  #60  in  MCAS  performance.  Nwwood 
ranks  #90  demographically  and  #65  in  MCAS  performance.  Millbuiy  ranks  #122 
demographically  and  #85  in  MCAS  performance. 


predicted  scores  of  their  students.  Clinton's  total  score  was  52  ranks  above  its 
demographic  rank;  Ayer's  total  MCAS  score  was  43  ranks  above  its 
demography;  and  Gardner  was  24  ranks  above  its  demography.  Please  note 
that,  despite  challenging  demography  that  would  predict  lower-than-state- 
average  scores,  each  of  these  commtmities  had  to  substantially  exceed  its 
demography  to  score  above  state  average.* 

A  NOTEWORTHY  district  fits  the  first  specification  but  does  not  fit  the 
second.  Since  its  performance  helps  its  students  to  go  beyond  their 
demography,  it  is  still  worthy  of  note.  What  such  a  district  is  doing 
educationally  can  hold  viseful  lessons  for  districts  that  are  demographically 
similar,  but  do  not  outscore  their  demography.  Such  a  district  is  more  likely 
to  deliver  a  return  on  future  public  investment  than  an  ineffective  district  is. 
Here  Ludlow.  Leominster,  and  Everett  are  outstanding  examples. 

•  Ludlow  is  a  demographically-challenged  commimity  near  Springfield  in 
Western  Massachusetts.  Its  total  MCAS  scores  are  much  higher  (23  ranks) 
than  its  demography  would  predict 

•  Leominster  is  a  small  dty  on  Route  2  in  Central  Massachusetts.  Its  overall 
score  on  all  nine  MCAS  tests  is  22  ranks  higher  than  its  demography 
would  predict 

•  For  the  third  year  in  a  row,  Everett's  overall  score  on  all  nine  tests 
combined  is  higher  than  its  demography  predicts  (by  11  ranks).  Further, 
Everett's  Grade  4  scores  were  substantially  better  than  demography  would 
suggest  This  is  especially  heartening  in  that  Everett  is  a  very 
disadvantaged  dty  that  faces  many  challenges  in  moving  its  students  up  to 
high  standards.^ 


'  Clinton  ranks  #171  demographically  and  #119  in  MCAS  performance.  Ayer  ranks  #159 
demographically  and  #116  in  MCAS  performance.  Gardner  ranks  #180  demographically  and 
#156  in  MCAS  performance. 

*  Ludlow  ranks  #184  demographically  and  #151  in  MCAS  performance.  Leominster  ranks  #179 
demographically  and  #157  in  MCAS  performance.  Everett  ranks  #185  demographically  and 
#174  in  MCAS  performance. 


Basic  kinds  of  community  in  Massachusetts 


It  is  very  difficult  to  understand  how  well  we  are  doing  in  terms  of 
implementing  education  reform  imless  the  various  characteristics  of 
Massachusetts  commimities  are  factored  into  the  evaluation.  The 
Effectiveness  Index  research  is  based  on  a  methodology  that  statistically 
analyzes  demographic  data  so  that  individual  school  districts  can  be  compared 
to  their  demographic  peers.  For  purposes  of  this  report,  I  will  place  the 
Commonwealth's  school  systems  into  four  categories:  Advantaged 
Massachusetts;  Middle  Massachusetts;  Challenged  Massachusetts;  and  Fifteen 
Qties,  a  subset  of  Challenged  Massachusetts.  Each  of  these  groupings  is  based 
on  the  demography  of  their  communities.**' 

Advantaged  Massachusetts  commimities  (the  top  50  in  the  state's 
demography)  are  characterized  by  high  education  levels,  high  incomes,  and 
high  property  values.  Weston,  Hingham,  Franklin  and  Natick  are  part  of 
Advantaged  Massachusetts.  In  terms  of  educational  characteristics,  per-pupil 
expenditures  (PFE)  are  generous,  averaging  $6,414,  with  a  range  of  $10,265 
(Dover-Sherbom)  to  $4,920  (Westford)  (These  nimibers  are  Fiscal  Year  1998 
figxires.  Current  spending  is  higher].  State  education  reform  aid  tends  to  be 
low,  averaging  $496  per  year  per  pupil,  with  a  range  of  $1^43  (Franklin)  to 
$143  (Lincoln)."  There  are  very  few  students  (4%)  who  qualify  for  free  or 
reduced  Itmdv  a  measure  of  poverty.  The  statewide  average  for  Free/ Reduced 
Lunch  is  26%.  Almost  all  of  the  students  speak  English  as  a  primary  language. 
Parents  tend  to  have  been  successful  in  education,  and  they  tmderstand  the 
importance  of  good  schools. 


The  data  here  is  for  Grade  8.  I  am  using  Grade  8  (Cass  of  2004)  because  these  students  have 
spent  most  of  their  academic  careers  in  post-education  reform  classrooms  and  because  members  of 
the  Qass  of  2004  are  expected  to  pass  MCAS  in  order  to  graduate. 

The  amount  of  new  education  reform  aid  that  is  distributed  as  part  of  the  Education  Reform 
Act  of  1993  is  calculated  in  the  following  manner.  I  to<^  the  amount  of  Fiscal  Year  1994  Chapter 
70  funding  (state  education  aid)  and  divided  it  by  the  number  of  pupils  statewide.  I  then  took 
the  amount  of  FY  1999  Chapter  70  funding  and  divided  that  figure  by  the  niunber  ctf  students. 
The  difference  between  the  two  produces  a  per-pupil  figure  that  represents  the  new  education 
reform  aid.  Chapter  70  data  is  available  from  the  Massachusetts  Department  of  Revenue. 


Middle  Massachusetts  consists  of  about  140  communities  (the  ntunber  varies 
with  grade  due  to  the  fluctuating  number  of  regional  systems  as  grade  level 
changes)  clustered  around  the  demographic  center  of  the  state.  These  are 
typical  Massachusetts  hometowns,  where  average  is  the  norm.  Middle 
Massachusetts  is  home  to  commtmities  like  Norwood,  Beverly,  Abington, 
and  South  Hadley.  These  districts  average  a  $6,015  per-pupil  expenditure. 
PPE's  range  from  $10,814  in  Cambridge  to  $4,391  in  Winthrop.  Typically,  each 
receives  $882  per  year  per  pupil  in  new  education  reform  aid,  compared  to  the 
state  average  of  $1,263.  The  range  of  per-pupil  reform  aid  is  $3,067  (Mohawk 
Trail  Regional)  to  $63  (Somerset).  There  are  relatively  few  students  (12%)  who 
qualify  for  free  or  reduced  Itmch,  a  measure  of  poverty.  The  statewide  average 
for  Free/Reduced  Limch  is  26%. 

Challenged  Massachusetts  is  made  up  of  the  50  commtmities  at  the  lower  end 
of  the  state's  demography.  Districts  in  this  category  include  commtmities  such 
as  Htill,  Palmer,  and  Holbrook,  as  well  as  many  cities.  These  districts  average 
a  $6,100  per-pupil  expendittire.  Typically,  each  receives  $1,626  per  year  per 
pupil  in  new  education  reform  aid,  compared  to  the  state  average  of  $1,263. 
The  range  of  state  education  reform  aid  is  $3,643  (Lawrence)  to  $233  (Ayer). 
Many  students  (33%)  qualify  for  free  or  reduced  limch,  a  meastire  of  poverty. 
The  statewide  average  for  Free/ Reduced  Ltmch  is  26%. 

The  Fifteen  Cities  are  the  15  tirban  commtmities  that  are  the  most 
demographically  disadvantaged  of  the  50  commtmities  that  comprise 
Challenged  Massachusetts.  This  duster  includes  dties  such  as  Worcester, 
Boston,  Fitchburg,  and  Lawrence.  While  these  communities  are  induded  in 
Challenged  Massachtisetts,  there  are  three  good  reasons  to  focus  on  them  as 
part  of  the  Fifteen  Qties. 

1.  These  15  communities  educate  29%  -  almost  one-third  -  of  our  students. 
If  the  Massachtisetts  economy  is  to  retain  its  vitality,  and  we  are  to  offer 
qualify  education  for  all,  we  must  find  some  way  to  be  more  effective  at 
educating  the  300,000  or  so  students  in  these  systems. 
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Z  These  systems  have  large  percentages  of  students  failing  MCAS.  In  fact, 
29%  of  the  state's  students  attend  school  in  the  Fifteen  Cities,  but  82%  of 
the  students  who  failed  the  Grade  8  ELA  MCAS  and  56%  of  the  students 
who  failed  the  Math  Grade  8  MCAS  attend  school  in  these  15  systems. 


3.  They  are  remarkably  similar  in  their  demographic  profiles.  These  15 
districts  are  large  school  systems  that  are  home  to  45%  of  the  state's 
minority  students.  Since  minority  academic  performance  is  a  major 
concern  of  many  people,  reforming  these  systems  is  the  key  to  boosting 
minority  achievement. 


The  districts  in  the  Fifteen  Qties  average  a  $6,503  per-pupil  expenditure.  The 
range  is  $8,118  (Boston)  to  $5,876  (Fitchburg).  On  average,  each  receives  $2,447 
per  year  per  pupil  in  new  education  reform  aid,  compared  to  the  $1,263  state 
average.  The  range  of  state  aid  is  $3,643  (Lawrence)  to  $1,559  (Somerville). 
This  is  the  only  segment  of  the  state's  demography  where  a  majority  of 
students  (57%)  qualify  for  free  or  reduced  limch,  a  measure  of  poverty.  The 
statewide  average  for  Free /Reduced  lunch  is  26%. 

Characteristics  of  Basic  Kinds  of  Community 


PPE  98   Education    Free  Lunch    Percentage  Percentage  of  Percentage  ot 


Advantaged  Mass  $6,414 

Mid  Mass  $6,015 

Challenged  Mass  $6,100 

Fifteen  Cities  $6,503 


Refonn  Aid  EUgibility 


$496 
$882 
$1,626 
$2,447 


4% 
12% 
33% 
57% 


of  State 
Population 
25% 
26% 
49% 
29% 


Statewide 
ELASFAiL 

7% 

17% 

77% 

82% 


Statewide  IM8 
FAIL 
11% 
22% 
67% 
56% 


FFE  98  =  Fiscal  Year  1998  Per-pupil  expenditure 

Education  Reform  Aid  =  Per-pupil  education  reform  (Chapter  70)  aid  (See  Footnote  11) 
Ff«e  LuikIi  Eligibility  =  Percentage  of  students  eligible  for  Free  or  Reduced  Lunch 
Percentage  of  State  Population  -=  Percentage  of  overall  statewide  student  population. 
(Please  remember  Out  Fifteen  Cities  is  a  subset  of  Oudlenged  hAassadiusells  so  it  is 
not  a  separate  grouping  in  terms  of  adding  up  to  100%.) 

Percentage  of  ELA8  FAIL  >  Percentage  of  failing  students  statewide  w1k>  are  in  this 
group 

Percentage  of  M8  FAIL  *  Percentage  of  failing  students  statewide  failing  M8  MCAS 
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Overcoming  Demography's  challenges 


While  a  districf  s  demographics  have  much  to  do  with  the  educational 
performance  of  its  students,  some  districts  have  chosen  to  organize 
themselves  to  move  beyond  their  demography.  Several  systems  have 
demonstrated  that  commtmity  and  student  characteristics  need  not  be  the 
determining  factor  in  student  achievement  (See  Appendix  A  for  a  listing  of 
over-performers  on  the  2000  MCAS.) 

•  On  Grade  4  MCAS,  Ayer,  and  Everett  over-performed  in  ELA  4;  Ayer, 
Chelsea,  and  Orange  over-performed  in  Mathematics;  and  Ayer,  Everett, 
and  Orange  over-performed  in  Science  and  Technology. 

•  On  Grade  8  MCAS,  demographically-challenged  overachievers  include 
Gardner,  Lawrence,  and  Berkshire  Hills  (Great  Barrington,  Stockbridge, 
West  Stockbridge)  in  ELA;  Chelsea,  Lawrence,  Leominster,  Ludlow,  and 
Gardner  in  Math;  and  Gardner,  Lawrence,  Leominster,  and  Chelsea  in 
Science  and  Technology. 

•  On  the  Grade  10  MCAS  ELA,  systems  that  scored  beyond  their  demography 
included  Southbridge,  Clinton,  Chelsea,  Fitchburg,  Ayer,  and  HulL  On 
Grade  10  Math,  overachievers  induded  Ware,  Chelsea,  Ayer,  Greenfield, 
Webster,  and  Southbridge.  On  the  Science  and  Technology  tests, 
overachievers  included  Chelsea,  Clinton,  Southbridge,  Hull,  Ayer,  North 
Adams,  and  Fitchburg. 

All  of  the  above  substantially  over-achieved  their  demography  (3  scaled  score 
points  or  better)  despite  fodng  significant  demographic  diallenges. 

Making  the  Grade 

MCAS  and  the  Class  of  2004  Students  who  took  last  Spring's  MCAS  and  are 
members  of  the  Qass  of  2004  will  have  to  pass  MCAS  -  score  Needs 
Improvement  or  better  on  both  the  Math  and  English  Language  Arts  (ELA) 
tests  -  in  order  to  graduate.  Although  students  take  the  Histoiy  MCAS  and 
Science  and  Technology  MCAS  tests,  members  of  the  Class  of  2004  will  not 
have  to  pass  them  to  meet  state  requirements. 
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Let  us  imagine  that  the  scores  obtained  on  last  year's  Grade  8  MCAS  by 
students  in  the  Qass  of  2004  counted  for  graduation. 

•  For  students  who  attend  school  in  Advantaged  Massachusetts,  the  50 
communities  at  the  top  of  the  state's  demography,  98%  of  the  students  would 
have  passed  the  ELA  test  and  84%  would  have  passed  the  Math.  Since 
students  must  pass  both  to  graduate,  at  least  16%  of  the  students  in 
demographically  advantaged  communities  -  those  who  did  not  pass  Math  - 
would  have  failed  if  passing  last  year's  Grade  8  MG\S  were  a  graduation 
requirement. 

•  Concerning  students  who  live  in  Middle  Massachusetts,  the  140  or  so 
commtmities  arotmd  the  center  of  the  state's  demography,  94%  passed  last 
year's  Grade  8  ELA  MCAS.  In  Math,  69%  passed.  Since  students  must  pass 
both  ELA  and  Math,  at  least  31%  of  students  in  Middle  Massachusetts  -  those 
who  did  not  pass  Math  -  would  have  failed  if  passing  last  year's  Grade  8 
MCAS  were  a  graduation  requirement. 

•  For  students  who  attend  school  in  Challenged  Massachusetts,  the  50 
commtmities  in  the  lower  part  of  the  state's  demography,  86%  of  the  students 
would  have  passed  the  ELA  test  and  51%  would  have  passed  the  Math.  Since 
students  must  pass  both  to  graduate,  at  least  49%  of  students  in 
demographically  disadvantaged  commtmities  -  those  who  did  not  pass  Math 
-  would  have  failed.  Children  in  the  schools  in  the  lower  25%  of  the  state's 
demography  account  for  49%  of  all  of  the  public  school  students  in 
Massachusetts. 

•  For  students  who  attend  school  in  the  Fifteen  Cities,  the  most 
demographically  challenged  districts  in  the  state,  74%  passed  last  year's  Grade 
8  ELA  MCAS.  On  the  Grade  8  Math  test,  32%  of  students  in  the  Fifteen  Gties 
passed.  Since  students  must  pass  both  to  graduate,  at  least  68%  of  students  the 
Fifteen  Qties  -  those  who  did  not  pass  Math  -  would  have  failed.  Students  in 
the  Fifteen  Qtes  comprise  26%  of  all  of  the  public  school  students  in 
Massachusetts. 
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The  following  chart  illtistrates  the  range  of  achievement  between  advantaged 
and  disadvantaged  commtinities  in  Massachusetts  as  reflected  on  the  2000 
Grade  8  ELA  and  Math  MCAS. 


The  basic  message  of  this  chart  is  largely  unchanged  from  last  year.  Not 
surprisingly,  a  district's  success  in  placing  its  students  in  the  Pass  category  is 
very  dependent  upon  its  demography.  The  challenge  facing  education 
reformers  is  clear  as  we  assess  the  first  eight  years  of  the  contemporary  school 
improvement  effort:  Boost  the  achievement  of  students  in  our  less 
advantaged  conmumities. 
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THE  LESSONS  OF  MIDDLE  MASSACHUSETTS 


The  50  most  demographically  enriched  communities  of  Massachusetts  - 
Advantaged  Massachusetts  -  are  fine  places  to  live  and  attend  school.  On  the 
other  hand,  these  districts  may  not  have  much  to  teach  us  about  making 
public  education  more  effective.  Students  in  these  districts,  by  dint  of 
parental  involvement  and  positive  demographics,  are  well  prepared 
academically.  Just  about  all  of  them  do  well  on  just  about  any  test 

Communities  in  that  part  of  the  state  known  as  Middle  Massachusetts  are 
much  more  interesting  from  an  education  reform  perspective.   In  the 
demographic  ranking  of  Massachusetts  school  districts,  about  140  districts  are 
concentrated  in  the  demographic  center  of  the  state.^^  These  districts,  with 
over  two-and-a-half  million  people,  make  up  what  might  be  called  Middle 
Massachusetts.  They  are  the  every-towns  that  comprise  the  "typical" 
Massachusetts  commimity  -  Norwood,  Beverly,  Abington,  South  Hadley. 
These  systems,  which  are  essentially  demographically  neutral,  may  be  well 
suited  to  play  a  crucial  role  in  education  reform. 

For  the  state  as  a  whole,  demographic  differences  among  the  districts  explain 
83%  of  the  variation  in  the  districts'  average  overall  test  scores.  All  or  much 
of  the  other  17%  of  the  variation  is  probably  explained  by  the  differences  in 
how  the  school  districts  themselves  orgcuiize  their  educational  delivery 
systems. 

When  looking  at  either  end  of  the  demographic  ladder  (Advantaged 
Massachusetts  with  Weston^  Wellesley,  Medfield;  Challenged  Massachusetts 
with  Lawrence,  Lynn,  New  Bedford),  we  notice  that  scores  tend  to  be  very 
high  or  very  low.  While  there  is  some  variation,  the  solid  suburbs  score  well. 
Conversely,  the  cities  have  low  scores,  again  with  some  variation.  In  short, 
the  overwhelming  m<qority  of  students  in  advantaged  districts  pass  MCAS 
easily;  most  students  in  disadvantaged  districts  are  very  far  away  fix)m 
passing. 


Because  of  regional  systems,  the  number  of  Massachusetts  school  districts  varies  with  grade 
level  There  are  243  Grade  4  districts,  231  Grade  8  districts,  and  223  Grade  10  districts. 
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The  pattern  in  Middle  Massachusetts  is  different  Its  districts  exhibit  a  wide 
range  of  test  scores  ~  even  though  their  demography  is  relatively  similar. 
This  variation  can  be  seen  in  the  following  bar  graphs  that  show  total  MCAS 
scores  for  the  2000  MCAS  Grade  8  tests.  (I  am  using  the  Grade  8  tests  because 
these  students  are  members  of  the  second  dass  that  will  have  to  pass  MCAS 
in  order  to  qualify  for  a  high  school  diploma.) 

For  each  of  the  14  districts  in  the  demographic  middle  of  the  state  -  the  seven 
districts  above  the  demographic  mean  and  the  seven  below  the  mean  -  the 
tip  of  the  bar  represents  its  average  test  score  on  the  2000  MCAS.  While 
demography  for  this  group  is  a  constant^  scores  range  widely.  The  top  scaled 
score  of  these  14  districts  in  Grade  8  ELA  is  247  (Braintree);  the  lowest  score  is 
238.  In  Grade  8  Math,  the  top  scaled  score  is  239;  the  lowest  score  is  225.  A9- 
point  difference  in  ELA  scores  and  a  14-point  difference  in  Math  scores  in 
communities  that  are  very  similar  is  significant  in  that  top-scoring  systems 
may  well  have  lessons  to  teach  systems  that  have  not  done  as  well. 


MIDDLE  MASSACHUSEHS  GRADE  8  2000  MCAS 


210        215         220        225        230        235         240        245  250 

SCALED  SCORE 
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ELA8 

MATH  8 

Score 

Score 

Wobum 

243 

231 

Mariborough 

238 

225 

East  Bridgewater 

238 

229 

Falmoutli 

241 

228 

Old  Rochester 

242 

234 

Beverly 

245 

236 

Auburn 

243 

231 

Essex 

244 

229 

Silver  Lake 

242 

232 

Bellingham 

240 

226 

Hatfield 

241 

233 

Tyngsborough 

245 

239 

Braintree 

247 

236 

Norwood 

244 

234 

The  14  districts  in  this  chart  on  the 
previous  page  are  very  similarly 
demographically;  whether  a  district  is  at 
the  top  or  the  bottom  of  the  chart  is  not 
significant.   The  actual  scaled  scores 
achieved  by  each  on  the  2000  MCAS  is  set 
out  in  the  table  on  the  lefi. 


ELA8  MATHS 

FAIL%  FAfL% 

Wobum  5  29 

Marlborough  14  44 

East  Bridgewater  13  31 

Old  Rochester  4  22 

Falmouth  7  37 

Beverty  3  25 

Auburn  5  28 

Essex  0  31 

Silver  Lake  6  30 

Bellingham  8  40 

Hatfield  3  27 

Tyngsborough  1  14 

Braintree  3  23 

Norwood  4  26 


The  table  on  the  left  sets  out  the 
percentage  of  students  in  Fail  for  ELA 
8  and  Math  8  for  each  the  14  Middle 
Massachusetts  Communities 
referenced  in  the  text.  This 
information  is  also  shown  on  the 
chart  on  the  next  page. 


30 


The  range  in  the  percentage  of  students  who  failed  the  2000  ELA  MCAS 
varied  from  1%  (Tyngsborough)  and  3%  (Braintree)  to  14%.  Similarly,  the 
range  of  Fail  in  Grade  8  Math  was  14%  (Tyngsborough)  to  33%,  again  a  wide 
performance  range  in  commtmities  that  are  very  similar  demographically. 
Middle  Massachusetts  school  systems  looking  to  improve  their  achievement 
might  look  to  Braintree,  Wobum,  Tyngsborough,  and  Beverly  for  guidance. 


MIDDLE  MASSACHUSETTS  GR  8  2000  MCAS  FAIL  PERCENTAGE 


Tyngsborough  g 
Bf«Mra« 


As  you  can  see,  there  is  wide  variation  in  actual  test  scores  and  fail  rates  in 
towns  that  are  very  similar  demographically. 


Since  the  demographic  variation  is  slight,  but  the  viiriation  in  test  scores  is 
great,  this  pattern  suggests  that  much  of  the  variation  is  explained  less  by 
demography  than  by  differences  in  what  the  schools  of  Middle  Massachusetts 
are  doing. 
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Further,  the  test  scores  of  Middle  Massachusetts  districts  with  high  positive 
niunbers  on  the  Effectiveness  Index  are  just  as  high  as  the  scores  of  many  of 
the  advantaged  districts.  For  example,  for  Grade  4,  a  class  that  started  school 
after  the  Education  Reform  Act  of  1993  was  enacted,  the  total  MCAS  Grade  4 
scores  of  Woburn,  a  Middle  Massachusetts  district,  are  equal  to  the  scores  of 
Newton,  a  very  advantaged  commimity.  Similarly,  fourth  graders  in 
Arlington,  another  Middle  Massachusetts  district,  scored  one  point  better 
than  students  in  Franklin  and  Milton,  two  demographically  advantaged 
districts.  It  is  important  to  note  that  Newton,  Franklin,  and  Milton  all  made 
the  Effective  School  Districts  list  for  2000;  these  systems  are  doing  a  good  job 
adding  value  to  the  educational  performance  of  their  students.  The  story  here 
is  how  well  schools  in  Wobum  and  Arlington  did  in  moving  their  students 
beyond  their  demographics. 


ELA  Grade  4: 2000  MCAS  Scaled  Scoies 
(ELA,  Math,  Science  and  Technology) 


DISTRICT 

English 

Math 

Science  and 

Total  MCAS 

Language  Arts 

Technology 

2000 

Woburn 

240 

245 

252 

737 

Newton 

239 

249 

249 

737 

Arlington 

238 

246 

252 

736 

Franklin 

238 

247 

250 

735 

Milton 

239 

248 

248 

735 

If  more  Middle  Massachusetts  districts  become  as  effective  as  Wobum  and 
Arlington,  then  more  Middle  Massachusetts  districts  will  see  improved 
student  achievement.  Because  Middle  Massachusetts  districts  are 
demographically  similar,  what  makes  for  effective  schools  in  these  districts  is 
more  likely  to  be  replicable  for  other  Middle  Massachusetts  districts  looking  to 
improve.  Thus,  Middle  Massachusetts  can  be  an  especially  fruitful  place  to 
seek,  and  expect  to  find  districts  whose  schools  are  well  positioned  to  help  all 
of  their  students  achieve  high  standards. 
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Student  ACfflEVEMENT  IN  DEMOGRAPHICALLY  CHALLENGED 
DISTRICTS 


Three  years  of  evaluating  school  district  effectiveness  with  respect  to 
demography  has  demonstrated  that  it  is  much  more  likely  that  an  upper-  or 
middle-demography  district  will  over-perform  its  demographically-predicted 
score.  Thus,  it  is  not  helpful  to  compare  cities  and  disadvantaged  towns  to 
more  middle-  and  upper-class  communities  when  assessing  performance. 
Lower  demography  commimities  should  be  placed  together  and  compared 
against  each  other  to  identify  systems  that  are  doing  an  exemplary  job. 

This  does  not  mean  that  standards  should  be  lowered  for  some  districts.  It 
does  mean  that  developing  the  kinds  of  schools  where  all  children  in  fact  do 
learn  will  require  more  creativity,  commitment,  and  time  when  dealing  with 
demographically  disadvantaged  communities. 

Developing  educational  strategies  that  can  overcome  the  impact  of 
disadvantageous  demography  is  a  major  challenge  for  many  districts.  This 
reality  tends  to  make  it  much  more  difficult  for  an  urban  district  or  a 
disadvantaged  town  to  score  above  predicted  levels.  Achieving  even  a  few 
points  above  the  demographic  prediction  may  indicate  positive  reform  efforts 
in  a  lower-demography  system.  A  disadvantaged  system  that  scores  at  state 
average  will  have  had  to  do  much  more  work  than  a  middle-class  system 
with  the  same  score. 

The  following  reflects  an  analysis  of  the  MCAS  performance  of  the  lower  50 
school  districts  as  sorted  by  demography.  This  research  combines  results  from 
ti\e  1998, 1999,  and  2000  MCAS.  The  scaled  scores  listed  below  reflect  total 
performance  on  all  three  MCAS  administrations.  (See  Apjjendix  E  for  more 
detail.) 

•   In  Grade  4  ELA,  Ayer  and  Everett  scored  9  scaled  score  points  above  each 
district's  predicted  performance  level.  They  also  scored  above  the  state 
average. 
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•  In  Grade  4  Math,  Ayer  and  Everett  scored  15  points  above  predicted  score, 
and  both  districts  scored  at  or  above  state  average.  Orange  and  Clinton 
were  also  strong  performers. 

•  In  Grade  4  Science  and  Technology,  Orange  scored  14  points  above 
predicted  level  Everett  scored  12  over  prediction,  and  Ayer  scored  11 
points  above  the  demographicaUy  predicted  score.  All  three  systems  were 
above  state  average. 

•  In  Grade  8  ELA,  Clinton  scored  10  points  above  predicted  score,  Berkshire 
Hills  scored  9  points  over  predicted  level,  and  Gardner  scored  8  points 
over  prediction.  AU  of  these  systems  scored  above  state  average. 

•  In  Grade  8  Math,  Ludlow  scored  14  points  and  Methuen  scored  8  points 
over  predicted  score,  with  both  systems  above  state  average. 

•  In  Grade  8  Science  and  Technology,  Gardner  and  Berkshire  HiUs  scored  10 
points  above  prediction  with  both  systems  scoring  above  state  average. 

•  In  Grade  10  ELA,  Southbridge's  actual  score  was  12  points  over  predicted 
score,  Fitchburg's  students  scored  9  above  prediction,  and  Hull  scored  6 
points  over  predicted  performance.  AU  three  districts  were  above  state 
average. 

•  In  Grade  10  Math,  Ayer  topped  the  demographically-challenged  over- 
scbrers  at  18  points  over  predicted  score.  Southbridge  and  Ware  scored  16 
over  predicted  level.  All  three  were  above  the  state  average. 

•  In  Grade  10  Science  and  Technology,  Webster,  North  Adams,  and  Clinton 
scored  9  points  over  predicted  level  with  all  three  scoring  above  the  state 
average. 
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Large  Urban  Systems 


Conspicuous  in  their  absence  from  the  lists  of  schools  that  over-perform 
demography  are  most  large  uiban  districts.  Lawrence  and  Chelsea,  two  very 
disadvantaged  systems  that  have  developed  collaborations  with  the  state 
Department  of  Education  and  Boston  University  respectively,  are  recognized 
in  this  research  for  producing  scores  that  exceed  demography  (see  following 
section).  There  are  few  other  major  urban  districts  that  are  identified  for 
noteworthy  performance. 

This  is  not  a  surprise  in  that  a  basic  finding  of  this  research  is  that 
demographically  challenged  systems  have  a  much  more  difficult  time  in 
exceeding  demographic  expectations  than  do  middle-class  or  advantaged 
districts.  (See  page  5).  A  superintendent  in  a  district  like  Lowell  or  Worcester 
faces  a  much  more  difficult  challenge  in  moving  students  up  to  meetii^  high 
standards  than  does  a  superintendent  in  a  district  like  Beverly  or  Dedham. 
Despite  this,  there  are  some  large  urban  systems  that  have  demonstrated  a 
capacity  to  move  beyond  demography. 

The  most  encouraging  work  has  been  done  in  Grade  4.  These  children  started 
school  after  education  reform  had  been  enacted.  They  are  young  and 
susceptible  to  improved  classroom  organization  and  more  effective  pedagogy. 
Unfortunately,  urban  students  in  Grade  8  and  especially  in  Grade  10,  have 
spent  mudi  of  their  academic  careers  in  classrooms  where  meeting  high 
standards  was  not  a  priority.  Therefore,  Grade  4  and  earlier  is  where  many 
savvy  superintendents  have  concentrated  their  reform  resources. 

In  ELA  4  on  the  2000  MCAS,  Springfield,  Chelsea,  Holyoke,  Worcester,  and 
New  Bedford  have  shown  the  most  over-performance  of  m^'or  tirban 
systems.  In  Math  4,  Chelsea,  Lawrence,  Worcester,  and  Springfield  have 
exceeded  demography.  And  in  Grade  4  Science  and  Technology,  Chelsea, 
Lawrence.  Fall  River,  Worcester,  and  Springfield  have  moved  student 
performance  beyond  their  commimity's  characteristics.  The  overscores  for 
these  systems  are  relatively  modest  when  compared  to  those  of  middle-dass 
and  advantaged  systems,  but,  since  the  challenge  is  so  much  greater  in  urban 
districts,  the  gains  are  important.  (See  Appendix  E.) 
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Lawrence,  Chelsea,  and  Holyoke 

Lawrence,  Chelsea  and  Holyoke  are  the  three  most  demographically 
disadvantaged  communities  in  the  Conmionwealth-  They  are  in  a  class  by 
themselves  when  it  comes  to  the  challenges  they  face  each  day  in  overcoming 
demographic  disadvantage.  Chelsea  schools  have  been  in  partnership  with 
Boston  University  for  many  years,  and  the  Massachusetts  Department  of 
Education  has  taken  a  special  interest  in  the  Lawrence  public  schools. 

Lawrence  and  Chelsea  tend  to  be  identified  by  this  research  as  over- 
performing  in  terms  of  their  demography.  These  systems  are  not  designated 
Effective  because  they  do  not  score  at  or  above  state  average,  a  requirement  for 
an  Effective  School  District  in  this  report  This  year,  as  was  the  case  in  the 
past  two  reports,  Chelsea  and  Lawrence  are  among  those  challenged  systems 
that  over-perform  on  MCAS.  The  fact  that,  even  with  a  strong  over- 
performance  over  their  demographically-predicted  scores,  they  are  still  so  far 
below  even  state  average,  is  a  measure  of  the  enormity  of  the  challenge  that 
faces  urban  educators. 

Holyoke,  with  similar  demography,  does  not  fare  as  well  as  Lawrence  and 
Chelsea  in  its  quest  to  add  value  to  the  learning  readiness  of  its  children- 
Holyoke  has  not  had  special  attention  paid  to  it  by  the  Commonwealth,  nor 
has  it  been  able  to  collaborate  with  an  outside  higher  education  institution  to 
help  improve  education.  While  neither  Lawrence  nor  Chelsea  is  even  close 
to  creating  schools  that  are  successful  in  helping  their  children  reach  basic 
standards,  they  do  fare  better  than  Holyoke.  The  extra  help  received  by 
Lawrence  and  Chelsea  has  helped  their  situations,  although  each  district  has 
miles  to  go  before  it  even  begins  to  meet  the  goal  of  ensuring  that  all  children 
learn. 
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THE  IMPORTANCE  OF  READING  AND  WRITING 


While  students  need  to  develop  a  broad  range  of  skills  and  competencies^ 
reading  and  writing  are  the  foundations  for  academic  success.  E.  D.  Hirsdv  Jr., 
author  of  The  Schools  We  Need  and  the  developer  of  the  Core  Knowledge 
Curriculum,  observes  that "...  good  competency  tests  in  reading  turn  out  to  be 
powerfully  indicative  of  achieved  abilities  that  go  far  beyond  reading." 
(Hirsch,  Education  Week,  Feb  Z  2000;  p,  40  in  "The  Tests  We  Need,"  p,  64) 
Veteran  educators  understand  that  reading  is  the  linchpin  of  academic 
success. 

Having  solid  reading  and  writing  skills  are  necessaiy  conditions  for  doing 
well  on  the  MCAS  tests.  This  is  true  even  of  the  tests  in  mathematics.  Many 
of  the  problems  on  the  mathematics  tests,  particularly  in  grade  eight  and  ten, 
are  word  problems.  You  cannot  understand  these  problems  if  you  cannot 
understand  the  words.  In  all  subjects,  moreover,  many  questions  call  for  a 
written  answer,  as  short  as  a  sentence  or  two  or  as  long  as  an  essay  of  several 
paragraphs. 

The  tables  in  Appendix  A  lists  Effective  and  Noteworthy  districts  in  terms  of 
student  reading  performance  by  grade  level  on  MCAS  2000  (See  ELA  scores  by 
grade).  Several  interesting  districts  that  are  effective  in  teaching  reading 
(ELA)  in  Grade  4  are  Eastham,  Wobum  and  Ayer.  Each  of  these  districts  had 
MCAS  scores  stibstantially  above  their  predicted  value  and  above  state 
average.  In  Grade  8,  Bedford,  Nauset  regional,  Brookline,  and  Gardner  solidly 
over-scored  their  demography  in  ELA  and  scored  above  state  average. 

Grade  10  presents  an  interesting  situation.  Some  people  say  that  today's  high 
school  students  tmderstand  that  MCAS  does  not  co\mt  towards  graduation  so 
they  do  not  invest  significant  personal  resources  in  doing  their  best  on  the 
test  Time  will  tell  if  that  is  correct,  but  it  is  still  important  to  recognize  those 
systems  that  do  perform  beyond  demographic  expectations.  Grade  10  over- 
performers  in  English  Language  Arts  indude  Hatfield,  Stoneham,  Harvard, 
Shrewsbury,  and  Southbridge. 
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EDUCATION  REFORM  IN  M  ASSACHUSETTSTN  2001 


The  Education  Refonn  Act  of  1993  provides  an  opportunity  to  transform  our 
schools.  The  Act  can  be  understood  in  terms  of  two  basic  changes  it  brought 
about: 

•  Sharply  increasing  the  amount  of  state  aid  targeted  at  disadvantaged,  low- 
spending  communities;  and 

*  The  establishment  of  statewide  academic  performance  standards  (and 
curriculum  frameworks)  and  an  assessment  device  to  measure  progress 
towards  meeting  those  standards  (the  MCAS). 

The  desired  outcome  of  all  of  this  is  improved  student  achievement.  The 
annual  MCAS  statewide  assessments  are  one  way  to  gauge  changes  in  student 
performance  over  time.  MCAS  is  only  given  to  a  few  grades  (currentiy  4, 8 
and  10  with  changes  on  the  way),  and  it  cannot  replace  local  assessments. 
Districts  concerned  about  helping  all  children  learn  should  use  other  tests  to 
supplement  MCAS  results.  For  example,  a  district  that  employs  the  Stanford 
9  for  its  local  assessments  shoxild  set  up  a  process  to  track  student  progress 
year-to-year. 

Seven  years  into  the  endeavor,  there  has  been  relatively  little  research  done 
on  the  impact  of  the  current  education  reform  effort  We  do  have  some 
information  most  of  which  is  from  First  Findings  (dted  below): 

•  We  know  that  billions  of  dollars  in  new  state  school  aid  has  been 
distributed.  Some  dte  a  figure  of  $6  billion;  others  dte  a  lower  number, 
generally  around  $3  billion.  (The  explanation  for  the  different  ntmibers 
reflects  how  one  calculates  tiie  increased  aid.  Hiere  has  been  over  $6  billion 
in  new  state  aid  since  1993,  but  about  half  of  that  would  have  been  distributed 
anyway.  The  additional  $3  billion  or  so  is  new  funding  that  was  mandated  by 
ttie  Education  Reform  Act  of  1993.) 

•  Sdiool  districts  have  attained  the  target  of  100%  of  their  foundation 
budgets  on  schedule.  The  new  funding  was  intended  to  give  each  district  a 
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basic  foundation  level  of  resources.  That  has  happened.  (See  First  Findings: 
The  Summative  Report  of  the  Educational  Management  Audit  Board, 
Massachusetts  Department  of  Revenue,  Division  of  Local  Services,  Maixdi 
2000.) 

•  Most  districts  have  hired  additional  teachers,  and  teacher  salaries  are 
up.  First  Findings  found  that  for  the  subset  of  districts  considered,  teacher 
salaries  rose  about  6%  faster  than  inflation.  The  increases  for  veteran 
teachers  was  more,  with  real  salaries  growing  by  roughly  12%.  (See  A  n 
Update  on  School  Reform  in  Massachusetts  by  Thomas  J.  Kane,  Associate 
Professor  of  Public  Policy,  John  F.  Kennedy  School  of  Government,  Harvard 
University,  which  was  done  in  conjunction  with  f  irsf  Findings.) 

On  the  other  side  of  the  ledger 

•  Costs  related  to  special  education  services  absorb  a  disproportionate 
amount  of  the  new  financial  resources. 

•  Most  districts  have  not  utilized  governance  changes  establishing 
performance-based  contracts  for  principals. 

•  While  nearly  all  school  districts  have  created  school  improvement 
plans,  many  of  these  plans  do  not  address  student  performance  or  test  results. 

•  There  is  so  far  little  correlation  between  the  increased  investment  in 
the  dassroom  as  measured  by  per-pupil  net  school  spending  and 
achievement. 

The  message  of  First  Findings  was  that  while  some  changes  have  occurred, 
most  school  cultures  and  school  systems  have  not  fundamentally  reorganized 
themselves  to  boost  student  achievement  The  post-reform  education 
delivery  system  in  Massachusetts  is  essentially  the  same  as  it  was  pre-reform. 
Since  March  2000,  when  First  Findings  came  out,  we  have  had  another  MCAS 
administration  that  shows  relatively  little  change  from  the  first  two 
administrations.  This  past  stunmer,  Boston  and  Maiden  made  an  effort  to 
change  teacher  contracts  to  make  schools  more  effective  in  teaching 
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students.  In  both  cases,  the  teacher  unions  essentially  rebuffed  the  reform 
e^orts.  Indeed,  the  rallying  ay  at  contract  negotiation  time  was  that  proposed 
changes  were  "anti-teadier"  and  showed  "a  lack  of  respect."  Thus,  as  the  new 
century  begins  with  the  challenge  of  doing  a  better  job  of  educating  all 
children,  the  work  rules,  which  govern  virtually  every  aspect  of  how  schools 
are  operated  in  Maiden  and  Boston  and  other  districts,  are  basically 
imchanged  from  what  they  have  been  for  many  years. 

Looking  Ahead 

Public  policy  is  written  in  bold  strokes.  Of  necessity,  a  law  or  a  program  must 
be  drawn  broadly  enough  to  cover  all  contingencies.  Once  a  public  initiative 
is  in  operation,  however,  the  game  may  change.  Seven  years  after  passage  of 
the  Education  Reform  Act  of  1993, 1  would  suggest  that  several  points  have 
emerged. 

•  Education  reform  is  not  a  singular  enterprise.  Given  the  vast 
diversity  of  the  Massachusetts  educational  commimity,  a  school 
improvement  plan  in  a  town  like  Natick  must  be  quite  different  from  a  plan 
in  a  dty  such  as  Taimton.   Most  of  our  students  in  middle-class  and 
advantaged  districts  are  already  within  relatively  easy  distance  of  passing 
MCAS.  Most  of  our  students  in  demographicaUy  challenged  urban 
commtmities  are  not.  The  remedy  for  the  cities  will  be  quite  different  from 
the  prescription  for  the  towns. 

It  is  dear  that  the  dties  need  a  more  robust  education  reform  effort  Seven 
years  and  three  billion  dollars  into  this  effort,  the  general  reform  tools 
provided  by  the  Education  Reform  Act  of  1993  do  not  appear  to  be  strong 
enough  to  make  a  difference  to  the  educational  success  of  urban  youth. 

The  challenge  in  Middle  Massachusetts  and  Advantaged  Massachusetts  is  to 
move  students  up  to  Profident  and  Advanced.  That  is  a  far  different  quest 
than  providing  a  young  person  in  Boston  or  Worcester  with  the  basic  literacy 
and  math  tools  needed  to  have  a  good  chance  for  success  in  life. 
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•  We  do  not  know  how  well  we  are  doing.  There  is  a  remarkable  lack 
of  evaluation  of  education  reform  in  Massachusetts.  Less  than  one-quarter  of 
one  percent  of  state  education  reform  funding  has  been  used  to  evaluate  how 
well  we  are  spending  the  other  99.75%  of  the  money.  The  state  Department  of 
Education  recentiy  developed  School  Performance  Rating  Reports.  The 
Massachusetts  Education  Reform  Review  Commission  has  done  some 
thoughtful  work-  The  Department  of  Revenue's  Education  Management 
Accountability  Board  has  done  the  most  extensive  reports.  The  best  single 
effort  is  First  Findings  (dted  above),  but  that  enterprise  was  never  intended  to 
be  a  definitive  evaluation  of  the  current  school  improvement  effort  It  is  not 
productive  to  point  fingers  about  which  agency  should  have  done  this  or  that 
What  is  needed  is  to  jump-start  a  careful  and  sustained  analysis  of  education 
reform  in  Massachusetts. 

•  We  need  to  improve  the  schools.  With  a  booming  business  sector  for 
most  of  the  late  1990s,  some  of  the  structural  shortcomings  of  the 
Massachusetts  economy  may  have  been  hidden.  Living  in  Massachusetts  is 
very  expensive;  the  climate  is  very  cold.  The  Commonwealth  does  not  attract 
many  people  from  out-of-state  into  the  work  force.  For  Massachusetts  to 
sustain  its  economic  viability,  we  will  have  to  find  a  way  to  find  productive 
employees  in-state.  Thousands  of  these  potential  workers  now  attend  urban 
schools  that,  as  of  today,  are  not  preparing  them  for  success  in  life  or  in  the 
job  maricet  If  we  are  to  continue  to  set  the  standard  for  economic  success, 
then  we  must  reorganize  urban  sdiools  to  be  effective  in  educating  urban 
youth. 

After  three  years  of  MCAS,  we  do  see  slow  progress.  A  pace  of  incremental 
change  that  may  be  acceptable  in  a  middle-class  district,  where  just  about 
everyone  passes  MCAS  now,  is  not  good  enough  for  a  system  with  high 
percentages  of  students  who  do  not  possess  basic  skills  and  who  fail  MCAS. 

•  MCAS  did  not  create  the  reality  of  thousands  of  students  who  cannot 
read,  write,  or  do  math  at  a  basic  level.  MCAS  has  only  identified  the 
underlying  reality  that  many  students  in  many  districts  are  awarded  a  high 
school  diploma  without  possessing  the  basic  skills  expected  of  a  high  school 
graduate.  MCAS  is  unique  in  Massachusetts  educational  history  in  that  it 

41 


is  the  first  program  that  tests  all  systems  every  year.  Pre-MCAS,  each  school 
system  cotild  choose  its  own  assessment  tool  and  declare  victory  based  on  the 
results.  As  a  Boston  public  school  parent  in  the  1980s,  I  was  routinely  assured 
that  students  were  learning.  As  a  Massachusetts  citizen  of  the  1990s  with 
access  to  MCAS  data,  I  know  that  those  statements  may  have  been  a  tad 
inaccurate.  MCAS  has  leveled  the  playing  field  in  terms  of  assessments.  We 
now  know  how  well  our  cities  are  doing  relative  to  our  middle  class  towns 
relative  to  our  advantaged  commimities. 

MCAS  is  a  truth-teller  of  sorts  in  tiiat  it  assesses  all  systems  every  year  using 
the  same  scoreboard  and  rules.  That  is  a  unique  contribution  to  our 
understanding  of  how  well  our  schools  are  doing.  People  of  good  faith  and 
sincere  heart  can  debate  various  aspects  of  MCAS,  but  in  the  absence  of  a 
means  to  evaluate  what  our  students  actually  know,  schools  and  districts  can 
declare  themselves  successful  in  educating  students  regardless  of  the  objective 
reality.  It  may  be  that,  had  MCAS  been  in  place  ten  or  twenty  years  ago,  we 
would  not  have  seen  the  routine  social  promotion  that  has  placed  thousands 
of  high  school  students  in  situations  where  it  will  be  very  difficult  for  them  to 
demonstrate  any  mastery  of  skills. 

End  Note 

This  study  captures  the  role  that  demography  plays  in  student  performance 
on  the  MCAS.  While  demography  is  not  destiny,  it  does  establish  a  tendency. 
If  we  overlook  the  tendency  of  disadvantaged  districts  to  produce  low  scores, 
then  we  will  continue  to  consign  the  children  of  those  districts  to  a  future  of 
unfulfilled  potential  After  seven  years  of  education  reform,  demography 
still  plays  too  large  a  role  in  the  school  performance  of  our  children.  In 
Massachusetts  in  2001,  where  you  happen  to  live  has  more  to  do  with  your 
success  in  school  than  any  other  factor. 
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APPENDIX  A 


EFFECTIVE  AND  NOTEWORTHY  SCHOOLDISTRICTS 
ONTHE2000MCAS 


The  following  tables  list  the  top  60  over-performing  school  districts  in  English 
Language  Arts,  Math,  and  Science  and  Technology  for  Grades  4,  S,  and  10. 
Over-performing  districts  score  higher  on  MG\S  than  would  be  predicted  by 
their  demographic  characteristics. 
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Effective  and  Noteworthy  Districts  on  the  2000  MCAS 


The  following  charts  list  the  top  sixty  districts  in  terms  of  their  score  on  the 
Effectiveness  Index.  The  Effectiveness  Index  is  the  difference  between  the 
likely  score  based  on  a  district's  demography  and  the  district's  actual  score  on 
the  MCAS.  Districts  are  listed  from  higher  to  lower  Effectiveness  Indexes. 

When  analyzing  anything  about  student  performance,  it  is  important  to 
understand  that  many  Massachusetts  communities  are  part  of  regional  school 
districts  and  that,  depending  on  grade  level,  a  conununity's  school  district 
may  change.  Thus,  the  Grade  4  results  reflect  evaluating  a  different  set  of 
school  districts  than  do  the  Grade  8  results  and  do  the  Grade  10  results.  The 
following  charts  represent  98%  of  the  state's  students.  Due  to  small  sample 
size,  those  districts  where  fewer  than  45-50  students  took  the  MCAS  are  not 
included. 

There  are  351  communities  in  Massachusetts;  there  are  202  school  districts 
whose  children  took  MCAS  tests  in  all  three  grades.  The  other  commimities 
are  part  of  various  regional  school  districts. 

Listings  are  provided  for  the  following  categories: 

•  Grade  4  performance:  English  Language  Arts 

•  Grade  4  performance:  Math 

•  Grade  4  performance:  Science  and  Technology 

•  Grade  4  performance:  all  three  subjects  combined  (English  Language  Arts, 
Math,  Science) 

•  Grade  8  performance:  English  Language  Arts 

•  Grade  8  performance:  Math 

•  Grade  8  performance:  Science  and  Technology 

•  Grade  8  performance:  All  three  subjects  combined  (English  Language  Arts, 
Math,  Science.  NOTE:  The  History  test  results  are  not  included  in  this 
analysis) 


•  Grade  10  perfonnance:  English  Language  Arts 

•  Grade  10  performance:  Math 

•  Grade  10  performance:  Science  and  TechnoJogy 

•  Grade  10  performance:  All  three  subjects  combined  (English  Language 
Arts,  Math,  Science) 


Key: 

SSGORE  is  the  scaled  socxeof  the  district  onthe  2000  MCAS  subject/grade  assessment  For 
example,  ELA4  is  English  Language  Arts  for  Grade  4;  M4  is  Math  for  Grade  4 ,  and  such . 

OVERSCX)RE  is  the  difference  between  the  demographically-predicted  score  and  the  actual 
2000  MCAS  score.  The  higher  the  Overscore,  the  more  a  district  exceeded  its  demographic 
characteristics. 

KOC  is  Kind  of  Community.  For  this  report,  there  are  (our  Kinds  of  Community.  KOC  1  is  a 
Demographically  Advantaged  commimity;KOC  2  is  a  Middle  Massachusetts  community;  KOC 
3  is  a  Demographically  Challenged  community;  and  KOC  4  is  one  of  the  Fifteen  Cities,  a  subset 
of  demographically  challenged  communities  (KOC  3)  which  indudes  the  most  demographic- 
ally disadvantaged  communities  in  the  Commonwealth.  See  page  19  in  the  text  for  more 
detail. 

Note:  Districts  in  UPPER-CASE  are  Noteworthy.  All  others  are  Effective. 
Noteworthy  districts  exceed  their  demographically-Iikely  score  but  score 
below  state  average.  Effective  districts  exceed  their  demographically-likely 
score  and  score  at  or  above  state  average. 
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DISTRICT 

ELA4  SSCORE 

OVERSCORE 

KOC 

Eastham 

239 

8 

2 

Wo  bum 

240 

7 

2 

Canton 

241 

6 

^  2 

Dan  vers 

239 

6 

2 

Georgetown 

239 

5 

2 

Kingston 

237 

5 

2 

Orleans 

239 

5 

2 

Ayer 

235 

5 

3 

Marshfield 

239 

5 

2 

Braintree 

238 

4 

2 

Somerset 

237 

4 

2 

Needham 

241 

4 

1 

Lenox 

237 

4 

2 

Hopkinton 

240 

4 

1 

Foxbo  rough 

238 

4 

2 

Pembroke 

237 

4 

2 

Brewster 

237 

4 

2 

Brookiine 

239 

4 

2 

Scituate 

239 

■4 

1 

Milton 

239 

3 

2 

Harvard 

238 

3 

2 

Belmont 

240 

3 

1 

Bourne 

235 

3 

2 

LAWRENCE 

222 

3 

4 

Plymouth 

235 

3 

2 

Arlington 

238 

3 

2 

Acton 

241 

3 

1 

Holbrook 

234 

3 

3 

Wilmington 

237 

3 

2 

Franklin 

238 

3 

1 

Shrewsbury 

237 

3 

2 

Everett 

232 

3 

3 

West  Bridgewater 

235 

3 

2 

Central  Berkshire 

234 

2 

2 

North  Attieborough 

236 

2 

2 

Mattapoisett 

236 

2 

2 

Swampscott 

238 

2 

1 

Beverly 

235 

2 

2 

Concord 

241 

2 

1 

Clinton 

233 

2 

3 

Rockport 

236 

2 

2 

Easton 

237 

2 

2 

Groton  Dunstable 

238 

2 

1 

Monson 

234 

2 

2 

Natick 

238 

2 

1 

Sharon 

238 

2 

1 

Pentucket  Regtonal 

236 

2 

2 

Westwood 

239 

2 

1 

Manchester 

238 

2 

1 

SPRINGRELD 

226 

2 

4 

Newton 

239 

2 

1 

Douglas 

233 

2 

2 

Rockland 

233 

2 

2 

Hoiyoke 

223 

2 

4 

Lynnfield 

238 

2 

1 

North  Reading 

237 

2 

2 

Abington 

234 

2 

2 

Belchertown 

233 

2 

2 

Hingham 

238 

2 

1 

WestBoylston 

236 

2 

2 

DISTRICT  MATH4SSC0RE  OVERSCORE 


OrlMnft 

252 

12 

Eastham 

246 

10 

Ayer 

241 

248 

Wobum 

245 

\JmK  DlUTTS 

245 

Shrewsbury 

247 

rUirvsra 

248 

Uxbddge 

242 

6 

Milton 

248 

6 

North  Reading 

247 

6 

Lenox 

244 

€ 

Franklin 

247 

5 

West  Bridgewater 

242 

S 

CHELSEA 

225 

5 

Rockland 

240 

S 

Foxbo  rough 

245 

5 

Neeonam 

zso 

9 

Boxborough 

252 

5 

Pembroke 

243 

5 

Arlington 

246 

5 

1  I  n  111  ■■in  n  1 

nolDtook 

9 

Danvers 

243 

5 

Nashoba 

246 

Northampton 

240 

ORANGE 

234 

Falmouth 

241 

Braintree 

243 

Brookline 

247 

Tyngsborough 

241 

Westborough 

246 

Westford 

248 

Newton 

249 

Manchester 

247 

Bourne 

240 

Marion 

239 

Plymouth 

240 

Georgetown 

243 

Everett 

235 

Mohawk  Trail 

238 

Sharon 

247 

Littleton 

244 

3 

Amherst 

241 

3 

East  Bridgewater 

240 

3 

Hampden  Wilbraham 

243 

3 

Dedham 

241 

3 

Clinton 

237 

3 

West  Boylston 

243 

3 

Stoughton 

240 

Easton 

244 

Wrentham 

242 

Scituate 

245 

LAWRENCE 

219 

Abington 

239 

2 

Beverly 

240 

2 

Central  Berltshire 

238 

2 

Belmont 

247 

2 

Wakefield 

242 

2 

Swampscott 

245 

2 

Southwick  Tolland 

241 

2 

DISTRICT 
Eastham 

Wobum 

Ayer 

Orleans 

Arlington 

Harvard 

Boylston 

Everett 

Marsftfield 

Orange 

Manchester 

Swansea 

Pembroke 

Westford 

Foxborough 

Hampden  Wilbraham 

Carfisle 

Brooldine 

Marion 

Amherst 

Braintree 

Kingston 

Carver 

Brewster 

Oedham 

North  Reading 

Need  ham 

Granby 

Topsfield 

Oak  Bluffs 

CHELSEA 

East  Longmeadow 

Hopkinton 

Franklin 

Rockland 

Deerfleid 

Falmouth 

Central  Berkshire 

Somerset 

HOLYOKE 

North  bridge 

Btackstone  Millville 

Plymouth 

Westborough 

Littleton 

West  Boylston 

Beliingham 

Mohawk  Trail 

Stoughton 

Berkley 

Grafton 

Easton 

LAWRENCE 

FALL  RIVER 

West  Bridgewater 

Lynnfield 

Bourne 

Acton 

QuabUn 

Nashoba 


ST4SSCORE  OVERSCORE 


2S3 

10 

252 

8 

249 

7 

254 

7 

252 

6 

253 

6 

252 

5 

244 

5 

251 

243 

252 

247 

249 

253 

250 

250 

256 

248 

4 

246 

3 

245 

3 

248 

3 

247 

3 

246 

3 

249 

3 

247 

3 

250 

3 

251 

3 

248 

3 

251 

3 

248 

3 

228 

3 

248 

2 

251 

2 

250 

2 

245 

2 

249 

2 

246 

2 

246 

2 

246 

2 

228 

2 

245 

2 

248 

2 

246 

2 

249 

2 

249 

2 

249 

2 

247 

2 

245 

2 

245 

2 

245 

2 

247 

2 

249 

2 

99A 

2 

235 

2 

246 

2 

246 

2 

252 

2 

246 

2 

249 

2 

GR4TOTAL 

OVcRSCORE 

KOC 

Eastham 

736 

28 

2 

Orleans 

745 

23 

2 

Wobum 

737 

22 

2 

Ayer 

725 

19 

-  3 

Marshfield 

738 

16 

2 

Harvard 

739 

16 

2 

Arlington 

736 

14 

2 

Foxborough 

733 

13 

2 

Pembroke 

729 

13 

2 

Need  ham 

742 

12 

1 

Bn  in  tree 

729 

12 

2 

Brookline 

734 

12 

2 

Shrewsbury 

732 

11 

2 

Lenox 

728 

11 

2 

Everett 

711 

11 

3 

Dan  vers 

728 

11 

2 

Franklin 

735 

11 

1 

Milton 

735 

10 

2 

North  Reading 

734 

10 

2 

ORANGE 

706 

10 

3 

Georgetown 

729 

10 

2 

West  Bndgewater 

723 

10 

2 

Manchester 

737 

10 

1 

Kingston 

722 

10 

2 

Rockland 

718 

9 

2 

CHELSEA 

676 

9 

4 

Plymouth 

721 

9 

2 

Bourne 

721 

9 

2 

Oak  Bluffs 

725 

9 

2 

Brewster 

727 

9 

2 

Uxbridge 

720 

8 

2 

Westford 

738 

8 

1 

Canton 

732 

8 

2 

LAWRENCE 

665 

8 

4 

Marion 

717 

8 

2 

Falmouth 

720 

8 

2 

Sc'rtuate 

733 

8 

1 

Hampden  Wiltnaham 

728 

7 

2 

Newton 

737 

7 

1 

Central  Berlcshire 

718 

7 

2 

Somerset 

721 

7 

2 

Easton 

730 

7 

2 

Westborough 

731 

7 

2 

Holbrook 

714 

7 

3 

Amherst 

718 

7 

2 

Littleton 

729 

7 

2 

Swansea 

717 

7 

2 

Sharon 

734 

7 

1 

West  Boylston 

728 

6 

2 

Belmont 

736 

6 

1 

Beverly 

721 

6 

2 

Stoughton 

719 

6 

2 

HOLYOKE 

673 

6 

4 

Nashoba 

730 

6 

2 

East  Longmeadow 

724 

6 

2 

Mohawk  Trail 

714 

6 

2 

Wilmington 

725 

6 

2 

Acton 

741 

6 

1 

Swampscott 

731 

5 

1 

Wrantham 

724 

6 

2 

4 


TOTAL  GR4 
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DISTRICT 

Bttdford 

Nauset 

Brookline 

Belohertown 

Gardner 

LAWRENCE 

Frontier 

Stoneham 

West  Bridgewater 

Central  Berkshire 

Berkley 

Berkshire  Hills 

Braintree 

East  Longmeadow 

Medfield 

Douglas 

Shrewsbury 

North  borough 

Carver 

Sandwich 

Harvard 

Need  ham 

Belmont 

Chatham 

Hanover 

Tyngsborough 

Leominster 

Westborough 

Beverly 

Quabbin 

Northampton 

Hull 

Westford 
Rockport 
Lenox 

North  Reading 

Berlin  Boylston 

SOUTHBRIDGE 

Nashoba 

Holbrook 

Lynnfiekl 

CHELSEA 

Tantasqua 

Quaboag  Regional 

Abington 

Canton 

Hampshire 

Clinton 

Dennis  Yarmouth 

Concord 

Edgartown 

Danvers 

Uxbridge 

Quinoy 

Norton 

Wachusett  Reg. 
Arlington 
West  Boylston 
Milfbrd 
Franklin 


ELA8  SSCORE    OVERSCORE  KOC 


254 

6 

1 

247 

5 

2 

249 

5 

2 

246 

5 

2 

242 

6 

3 

226 

5 

4 

246 

6 

2 

247 

5 

2 

246 

2 

245 

2 

245 

2 

243 

3 

247 

2 

247 

4  . 

2 

253 

4 

1 

244 

4 

2 

248 

4 

2 

250 

4 

1 

244 

4 

2 

247 

4 

2 

248 

4 

1 

251 

3 

1 

250 

3 

1 

246 

3 

2 

249 

3 

1 

245 

3 

2 

242 

3 

3 

248 

3 

2 

245 

3 

2 

244 

3 

2 

242 

3 

2 

242 

3 

3 

250 

3 

1 

246 

3 

2 

245 

3 

2 

248 

2 

2 

247 

2 

2 

237 

2 

4 

248 

2 

1 

242 

2 

3 

249 

2 

1 

227 

2 

4 

243 

2 

2 

241 

2 

2 

244 

2 

2 

248 

2 

2 

245 

2 

2 

241 

2 

3 

241 

2 

2 

252 

2 

i 

245 

2 

2 

245 

2 

2 

243 

2 

2 

242 

2 

3 

244 

2 

2 

247 

i 
I 

246 

2 

2 

246 

2 

2 

243 

2 

2 

247 

2 

1 

UIO  1  KlU  1 

Mo  OOUUKC 

UVtzKoUUKc 

ipswioh 

246 

11 

2 

Tynflsborough 

239 

8 

2 

Frontier 

240 

8 

2 

Rookport 

242 

8 

2 

Tantasqua 

237 

8 

2 

Westford 

248 

7 

1 

Harvard 

244 

7 

1 

Nauset 

238 

7 

2 

Grafton 

238 

7 

2 

Central  Berkshire 

235 

6 

2 

Chatham 

238 

6 

2 

North  borough 

246 

6 

1 

Westborough 

243 

6 

2 

Shrewsbury 

241 

6 

2 

Bedford 

247 

6 

1 

Newbury  port 

240 

6 

2 

Brookline 

245 

6 

2 

Carver 

233 

6 

2 

Hanover 

242 

5 

1 

Sutton 

238 

5 

2 

Amherst  Pelham 

239 

5 

2 

Beverly 

236 

5 

2 

Berkley 

233 

6 

2 

CHELSEA 

213 

5 

4 

West  Bridgewater 

234 

5 

2 

Wachusett  Reg. 

242 

5 

1 

LAWRENCE 

210 

5 

4 

Douglas 

232 

5 

2 

Dennis  Yarmouth 

232 

2 

Newton 

247 

1 

Sandwich 

239 

4 

2 

Peabody 

232 

4 

3 

Leominster 

230 

4 

3 

Ludlow 

229 

4 

3 

Braintree 

236 

2 

Carlisle 

257 

1 

Mohawk  Trail 

232 

2 

Stoneham 

235 

2 

GARDNER 

227 

3 

Milford 

234 

2 

Uxbridge 

233 

2 

Acton  Boxborough 

250 

4 

1 

Pentucket  Regional 

238 

2 

Edgartown 

238 

2 

BelnK>nt 

245 

1 

Amesbury 

233 

2 

Medway 

241 

3 

1 

Lexington 

248 

3 

1 

Hadley 

235 

3 

2 

Georgetown 

237 

3 

2 

Hampshire 

235 

3 

2 

REVERE 

223 

3 

4 

Northampton 

231 

3 

2 

EVERETT 

224 

3 

3 

Millbury 

231 

3 

2 

GREENRELD 

226 

3 

3 

Chelmsford 

241 

3 

1 

Gloucester 

229 

3 

2 

Cohasset 

249 

3 

1 

Quabbin 

232 

2 

2 

M8 


61 

I 


DISTRia 

ST8SSC0RE 

aw*>^^* 

OVERSCORE 

KOC 

WesttMrough 

245 

8 

2 

Tyngsborough 

240 

8 

2 

Bedford 

247 

8 

1 

East  Long  meadow 

241 

8 

2 

Quabbin 

239 

7 

2 

Quaboag  Regional 

235 

7 

2 

Leicester 

239 

6 

2 

Belmont 

244 

6 

1 

Nauset 

238 

6 

2 

Brookline 

239 

6 

2 

Sandwich 

241 

6 

2 

Hanover 

243 

6 

1 

Stoneham 

237 

6 

2 

Gardner 

231 

6 

3 

Belchertown 

236 

6 

2 

Masconomet 

244 

5 

1 

Central  Berkshire 

236 

5 

2 

Douglas 

235 

5 

2 

Frontier 

237 

5 

2 

Northampton 

233 

5 

2 

Monson 

236 

5 

2 

West  Bridgewater 

236 

5 

2 

Harvard 

241 

5 

1 

Carver 

235 

4 

2 

Hampden  Wilbraham 

239 

4 

2 

Norwell 

244 

4 

1 

Canton 

240 

4 

2 

Amherst  Pelham 

233 

4 

2 

Needham 

243 

4 

1 

LAWRENCE 

209 

4 

4 

Concord 

246 

4 

1 

Andover 

244 

4 

1 

North  Reading 

240 

4 

2 

Med  way 

241 

4 

1 

Chatham 

237 

4 

2 

Beverly 

235 

4 

2 

Berlin  Boylston 

239 

4 

2 

Hingham 

243 

4 

1 

West  Boylston 

238 

4 

2 

Shrewsbury 

238 

4 

2 

Silver  Lake 

236 

3 

2 

Rockport 

237 

3 

2 

Long  meadow 

244 

3 

1 

Millbury 

234 

3 

2 

Leominster 

230 

3 

3 

Westford 

243 

3 

1 

Lunenburg 

237 

3 

2 

Berkshire  Hills 

231 

3 

3 

Edgartown 

238 

3 

2 

Winthrx>p 

233 

3 

2 

CHICOPEE 

226 

3 

3 

Granby 

236 

3 

2 

Pentucket  Regional 

238 

3  ' 

2 

Tantasqua 

234 

3 

2 

Nantucket 

236 

3 

2 

Norwood 

235 

3 

2 

Braintree 

235 

3 

2 

Berkley 

233 

3 

2 

Sutton 

237 

3 

2 

Dennis  Yarmouth 

231 

3 

2 

niCTDI/*T 
Ulo  1  KIV«  1 

1 U 1  ALo  ooUUKC 

UVcKoOUKc: 

Bedford 

748 

20 

1 

Tyngsborough 

724 

20 

2 

Westborough 

796 

19 

2 

Nauset 

723 

18 

2 

Frontier 

723 

18 

2 

Brookline 

733 

17 

2 

Central  Berkshire 

716 

16 

2 

Harvard 

733 

15 

1 

Gardner 

700 

14 

3 

Stoneham 

719 

14 

2 

Hanover 

734 

14 

1 

Sandwich 

727 

14 

2 

Rockport 

725 

14 

2 

West  Bridgewater 

716 

14 

2 

Carver 

712 

14 

2 

East  Longmeadow 

722 

14 

2 

Douglas 

711 

13 

2 

UVWRENCE 

645 

13 

4 

Chatham 

722 

13 

2 

Belmont 

739 

13 

1 

Shrewsbury 

727 

13 

2 

Westford 

741 

13 

1 

Tantasqua 

714 

13 

2 

Quabbin 

715 

13 

2 

North  borough 

736 

12 

1 

Beverly 

716 

12 

2 

Berkley 

711 

12 

2 

Belchertown 

712 

12 

2 

Braintree 

718 

11 

2 

Ipswich 

724 

11 

2 

Leominster 

702 

11 

3 

Northampton 

706 

11 

2 

Amherst  Pelham 

712 

10 

2 

Quaboag  Regional 

702 

10 

2 

Leicester 

715 

9 

2 

CHELSEA 

650 

9 

4 

Dennis  Yarmouth 

704 

9 

2 

Wachusett  Reg. 

728 

9 

1 

Edgartown 

721 

9 

2 

North  Reading 

726 

9 

2 

Canton 

727 

8 

2 

Med%vay 

729 

8 

1 

Concord 

747 

8 

1 

Pentucket  Regional 

721 

8 

2 

West  Boytston 

720 

8 

2 

Need  ham 

737 

8 

1 

Hingham 

736 

8 

1 

Ne%vton 

735 

8 

1 

Sutton 

718 

7 

2 

Peabody 

704 

7 

3 

Berlin  Boytston 

723 

7 

2 

Berkshire  Hills 

701 

7 

3 

Mitfbrd 

709 

7 

2 

Uxbridge 

708 

7 

2 

Carlisle 

769 

7 

1 

Hadley 

712 

7 

2 

Grafton 

713 

7 

2 

Lexington 

742 

7 

1 

Hampshire 

716 

6 

2 

Abington 

709 

6 

2 

4 


TOTAL  GR8 
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DISTRICT 

ELA10  SSCORE 

OVERSCORE 

KOC 

1  1  iiif  ■  1  ri 
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Z49 
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uiinton 
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1  tfH»tnn 
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1 
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Draintree 

Z4o 
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1 
Z 
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Lenox 
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9 

1 

wuinsncKi 

Z41 

m 
9 

a 

2 

Gill  Montague 

Z97 

O 

a 
2 

K 
O 

4 

1 

Manchester 

243 

O 

4 

1 

Rockport 

240 

O 

a 
2 

wonassef 

e 
9 

4 
1 

Lee 

1U 

234 

a 
2 

westDorougn 

141 
242 

a 
2 

caston 

141 
243 

4 

1 
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OAK 
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1 

Frontier 

110 

23o 

a 

2 

nopKinxon 

4 

1 

woDum 

no 
23o 

a 
2 

uougias 

Zoo 

a 
Z 

Grafton 

no 
239 

a 
Z 

LynnneKi 

Z44 

4 

1 

Svnnsea 

11  c 

239 

a 
2 

Medway 

tAI 
Z4o 

4 

T 

Need  nam 

14  4 

244 

1 

Nortriampion 

235 

1 
3 

2 

wincnester 

147 

247 

3 

1 

Asniana 

141 

Z43 

4 

3 

1 

IA/mmL.  1 ,  .    mil  Dam 

wacnuseti  Keg. 

Z4l 

a 

3 

1 

Tyngsborough 

no 
Z3S 

a 
3 

Z 

oenin  ooyisvon 

Z4U 

a 

3 

a 

2 

ware 

i9<f 
Zol 

a 

3 

a 
3 

West  Boy  IS  ton 

239 

3 

2 

NOfin  Keaoing 

144 

Z41 

3 

1 

Norweii 

14C 

24o 

3 

1 

East  Bridgewater 

117 

237 

3 

2 

LAWKcNCc 

14  C 

21 0 

3 

4 

Sharon 

141 

243 

a 
3 

if 

1 

warefiam 

233 

3 

3 

Granby 

237 

3 

2 

Bourne 

11^ 
236 

3 

2 

Groton  Dunstable 

243 

3 

1 

Mlllbury 

235 

3 

2 

Glouoester 

233 

3 

2 

ELA10 
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DISTRICT 

MATH10  SSCORE 

OVERSCORE 

KOC 

Harvard 

264 

18 

1 

Lenox 

243 

11 

2 

Westborouflh 

249 

11 

2 

Lee 

236 

11 

*  2 

Ipswich 

246 

10 

2 

Nantucket 

244 

10 

2 

Nauset 

241 

10 

2 

Ware 

232 

9 

3 

Grafton 

242 

9 

2 

Shrewsbury 

243 

9 

2 

Brainlree 

241 

8 

2 

Georgetown 

243 

8 

2 

Hadley 

241 

8 

2 

Frontier 

239 

8 

2 

CHELSEA 

218 

8 

4 

Tyngsl>orough 

240 

8 

2 

Northt>oro  Southboro 

249 

7 

1 

Norweil 

250 

7 

1 

Norton 

239 

7 

2 

Berlin  Boylston 

243 

7 

2 

Ayer 

233 

7 

3 

Greenfield 

231 

7 

3 

Dedham 

238 

7 

2 

Martha's  Vineyard 

238 

7 

2 

Hudson 

239 

6 

2 

Mansfield 

241 

6 

2 

Wachusett  Reg. 

243 

6 

1 

West  Boylston 

240 

6 

2 

Hopkinton 

247 

6 

1 

Littleton 

242 

6 

2 

Newton 

248 

6 

1 

Belmont 

247 

5 

1 

Webster 

229 

5 

3 

Old  Rochester 

235 

5 

2 

Northampton 

234 

5 

2 

Milibury 

234 

5 

2 

Wobum 

237 

5 

2 

Southbridge 

234 

5 

4 

Acton  Boxborough 

251 

5 

1 

Dudley  Charlton 

236 

5 

2 

FITCHBURG 

224 

4 

4 

Reading 

245 

4 

1 

Westford 

245 

4 

1 

Sandwich 

239 

4 

2 

Clinton 

230 

4 

3 

Groton  Dunstable 

244 

4 

1 

Concord  Carlisle 

253 

4 

1 

Sharon 

245 

3 

1 

North  bridge 

230 

3 

2 

Stoneham 

235 

3 

2 

LOWELL 

221 

3 

4 

Lynnfield 

244 

3 

1 

Brookline 

243 

3 

2 

NORTH  ADAMS 

226 

3 

3 

Hull 

230 

3 

3 

Ludlow 

229 

3 

3 

Needham 

245 

3 

1 

East  Bridgewater 

234 

3 

2 

Westwood 

245 

3 

1 

Dighton  Rehotx>th 

235 

3 

2 

DISTRICT 

Westborough 

Harvard 

Shrewsbury 

Hadley 

CHELSEA 

Nauset 

Millbury 

Braintrae 

Stoneham 

Lenox 

Hudson 

Berlin  Boylston 

Tyngsborough 

Clinton 

Belmont 

Nantucket 

Westwood 

West  Boylston 

Manchester 

Grafton 

Hatfield 

Southbridge 

Douglas 

Lee 

Wachusett  Reg. 

Norton 

Hull 

Webster 

Belchertown 

Littleton 

Reading 

Ayer 

North  Adams 
FITCHBURG 
North  Reading 
Medway 

Northboro  Southboro 

Quincy 

Oedham 

Northbridge 

Dudley  Charlton 

Wobum 

East  Longmeadow 

Bedford 

Newton 

Norwell 

Ralph  Mahar 

Lexington 

Hopkinton 

Easton 

Frontier 

Gk>uoester 

Mansfield 

Ware 

Quaboag  Regional 

Uxbridge 

Swansea 

LOWELL 

NEW  BEDFORD 

Dighton  Rehoboth 


SSCORE 

OVERSCORE 

KOC 

244 

11 

2 

244 

11 

1 

240 

9 

2 

239 

"8 

2 

220 

8 

4 

238 

8 

2 

236 

7 

2 

237 

7 

2 

236 

7 

2 

237 

7 

2 

236 

6 

2 

239 

6 

2 

237 

6 

2 

232 

6 

3 

240 

6 

1 

237 

6 

2 

241 

5 

1 

237 

5 

2 

238 

5 

1 

236 

5 

2 

235 

5 

2 

228 

5 

4 

233 

5 

2 

231 

4 

2 

238 

4 

1 

235 

4 

2 

231 

4 

3 

229 

4 

3 

234 

4 

2 

237 

4 

2 

239 

4 

1 

231 

4 

3 

229 

4 

3 

225 

4 

4 

237 

3 

1 

238 

3 

1 

239 

3 

1 

229 

3 

3 

232 

3 

2 

231 

3 

2 

234 

3 

2 

233 

3 

2 

234 

3 

2 

239 

3 

1 

237 

3 

1 

240 

3 

1 

230 

3 

3 

240 

3 

1 

239 

3 

1 

237 

3 

1 

233 

3 

2 

230 

3 

2 

235 

3 

2 

228 

3 

3 

231 

3 

2 

232 

3 

2 

231 

3 

2 

221 

3 

4 

221 

2 

4 

234 

2 

2 

DISTRICT 

TOTAL10  SSCORE 

OVERSCORE 
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APPENDIX  B 
REPEATERS : 

EFFECTIVE  AND  NOTEWORTHY  SCHOOLDISTRICTS 
ON  BOTH  THE  1999  AND  2000  MCAS 


The  following  tables  list  the  school  districts  that  most  over-performed  their 
demographic  characteristics  on  both  the  1999  and  2000  MCAS  in  English 
Language  Arts,  Math,  and  Science  and  Technology  for  Grades  4,  8,  and  10. 
The  lists  represent  districts  that  app)eared  the  top  60  lists  for  both  MCAS 
administrations. 


Key: 

CX)MBINED  OVERSCORE  99-00  is  the  sum  of  the  district's  Oversoores  on  both  the  1999  and 
2000  MCAS.  Overscore  is  the  difference  between  the  demographically-predicted  score  and  the 
actual  district  MCAS  score.  The  higher  the  Combined  Overscore,  the  more  a  district  exceeded 
its  demographic  characteristics.  For  example,  ELA4  99-00  is  the  combined  score  for  English 
Language  Arts  for  Grade  4  or  the  1999  and  2000  MCAS;  M4  99-00  is  Math  for  Grade  4 ,  and  such. 
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APPENDIX  C 


Over- PERFORMING  SCHOOLDISTRICTSON 
THE  1998, 1999,  AND  2000  MCAS 
BASED  ON  COMBINED  O VERSCORE 


The  following  tables  list  the  top  30  over-performing  school  districts  in  English 
Language  Arts,  Math,  and  Science  and  Technology  for  Grades  4,  8,  and  10 
using  the  combined  results  of  all  three  MCAS  administrations.  Over- 
performing  districts  score  higher  on  MCAS  than  would  be  predicted  by  their 
demographic  characteristics. 


Key: 

Per-Pupil  Education  Reform  Aid  is  the  difference  between  the  amount  of  per-pupii  Chapter  70 
state  school  aid  provided  in  1994  and  the  amount  provided  in  1998.  It  is  a  measure  of  how  mudi 
additional  resources  education  reform  brought  to  individual  districts  during  that  period.  See 
Footnote  11  in  the  text  for  moredetail. 

COMBINED  OVERSCORE  x.j(  98-00  is  the  sumof  the  district  s  Overscores  on  all  three  MCAS 
administrations  in  three  sut^ects.  For  example,  ELA4  is  English  Language  Arts  for  Grade  4;  M4 
is  Math  for  Grade  4,  and  such.  Overscore  is  the  difference  between  the  demographically- 
predicted  score  and  the  actual  district  MCAS  score.  The  higher  the  Overscore,  the  more  a 
district  exceeded  its  demographic  characteristics. 
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ELA4 


DISTRICT  Per^upll  Education  Reform  Aid     COMBINED  OVERSCORE  EU4  98-00 


Wobum 

344 

16 

Eastham 

349 

16 

Pembroke 

1193 

13 

Orleans 

393 

12 

Harvard 

287 

11 

East  Long  meadow 

384 

10 

Lawrence 

3643 

10 

Lenox 

301 

9 

Ayer 

233 

9 

Dan vers 

337 

9 

Everett 

1785 

9 

Brooldine 

322 

9 

Franklin 

1843 

8 

Kingston 

1259 

8 

Shrewsbury 

552 

8 

North  Reading 

529 

8 

Marshfield 

1210 

8 

Monson 

1741 

8 

Orange 

3244 

8 

Canton 

334 

8 

Newton 

335 

8 

Sharon 

295 

8 

Braintree 

336 

8 

Central  Berkshire 

976 

8 

Scituate 

370 

7 

Needham 

315 

7 

Hadley 

363 

7 

Georgetown 

558 

7 

Arlington 

325 

7 

Wrentham 

1530 

7 
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M4 


DISTRICT  Per4>upil  Education  Reform  Aid       COMBINED  OVERSCORE 11114  98-00 


Orleans 

393 

29 

Harvard 

287 

25 

Oak  Bluffs 

347 

22 

Shrewsbury 

552 

21 

North  Reading 

529 

21 

Eastham 

349 

19 

Chelsea 

3318 

18 

Newton 

335 

18 

Wobum 

344 

18 

Lenox 

301 

17 

West  Bridgewater 

344 

17 

Ptymouth 

1146 

16 

Orange 

3244 

16 

Littleton 

355 

15 

Marion 

327 

15 

Marshfield 

1210 

15 

Ayer 

233 

15 

Everett 

1785 

15 

Pembroke 

1193 

15 

Lexington 

306 

15 

Clinton 

1933 

14 

East  Lonameadow 

384 

14 

Franklin 

1843 

13 

Winchester 

332 

13 

SturtMidge 

436 

12 

Amherst 

1546 

12 

West  Boylston 

473 

12 

Foxborough 

974 

12 

Wachusett  Reg. 

1797 

11 

Belmont 

318 

11 
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ST4 


DISTRICT  Per-Pupll  Education  Reform  Aid  COMBINED  OVERSCORE  ST4  984)0 


OrfMIIS 

399 

ZD 

g  ■  ■ill.  ■_,  ■ 

949 

zo 

Wobum 

344 

18 

rMivara 

Zo7 

IB 

Pembroke 

1193 

15 

Orange 

3244 

14 

Everett 

1785 

13 

North  Reading 

529 

13 

Boylston 

94o 

19 

Arlington 

9ZO 

IZ 

Monson 

1741 

12 

Ayer 

233 

12 

Amtierst 

1o4o 

12 

Manon 

327 

12 

Oak  Blufre 

347 

12 

BrooMine 

322 

11 

Plymoutti 

1146 

11 

Hadiey 

363 

11 

Norwood 

342 

11 

Quabbin 

1359 

10 

East  LxMigmeadow 

384 

10 

Brewster 

421 

10 

Lenox 

301 

9 

Hampden  Wilbraham 

1470 

9 

Ne%vton 

335 

9 

Westford 

605 

9 

Chelsea 

3318 

8 

Granby 

819 

8 

Braintree 

336 

7 

Mohawk  Trail 

3067 

7 

68 


TOTAL  GR4 

DISTRICT  Per-Pupil  Education  Reform  Aid     GR4  TOTAL  MCAS  98-00 


Orleans 

393 

67 

Eastham 

349 

60 

Harvard 

287 

65 

Wobum 

344 

63 

Pembroke 

1193 

43 

North  Reading 

529 

42 

Orange 

3244 

38 

Everett 

1785 

37 

Lenox 

301 

36 

Ayer 

233 

36 

Newton 

335 

35 

Plymoiith 

1146 

34 

East  Longmeadow 

384 

34 

Oak  Bluffs 

347 

32 

Marion 

327 

31 

MarshfieM 

1210 

30 

Shrewsbury 

552 

30 

Chelsea 

3318 

29 

Arlington 

325 

29 

Monson 

1741 

29 

Franklin 

1843 

29 

Brookline 

322 

28 

Amherst 

1546 

28 

West  Bridge%vater 

344 

27 

LIUJelon 

355 

27 

Norwood 

342 

26 

Brewster 

421 

24 

Westford 

605 

24 

Lexington 

306 

24 

Lawrence 

3643 

24 

ELA8 


DISTRICT 

Per-PuDil  Education  Reform  Aid 

COIMBINED  OVERSC 

1  f 

H^ffVAfftl 

287 

14 

1  aufffmnrLii 

13 

Diwwffuinc 

10 

Frontier 

m  O 

1 Z 

SUA 

12 

976 

1  A 

Bedford 

318 

w  1 0 

12 

4^01; 
1499 

IZ 

o«iiHJwicn 

IZ 

Draiiiucc 

000 

n 

^000 

44 

11 

Nortluiffnptofi 

•fi 
1  1 

«v9 

lU 

4n 

lU 

1940 

lU 

wTODUni 

Q 

9 

ovifUHiifv  riiii9 

0 

Owl 

0 

9 

nonnDorougn 

9 

naaicy 

Ooo 

ft 

DdWf  ffW  WWII 

1  1 

a 
0 

Nonwood 

342 

8 

Gardner 

2016 

Medway 

1074 

Nauset 

1021 

Medfield 

332 

Chelsea 

3318 

Uxbndge 

1899 

70 


M8 


District  Per-Pupil  Education  Reform  Aid     COMBiNED  OVERSCORE  M8  98-00 


Harvard  287  31 

Ipswich  322  24 

Tyngsborough  1495  23 

Hadley  363  19 

West  Bridflewater  344  19 

Sandwich  326  18 

Frontier  473  18 

Medway  1074  18 

North  Reading  529  18 

Amherst  Peiham  1120  17 

Chelsea  3318  17 

Stoneham  319  17 

Newbury  port  315  17 

Tantasqua  1396  17 

Lawrence  3643  16 

Wesffoorough  309  16 

Central  Berkshire  976  15 

Grafton  807  15 

Northborough  328  14 

Chatham  306  14 

Westford  605  14 

Newton  335  14 

Beverly  385  14 

Ludlow  1304  14 

WachusettReg.  1797  13 

Brookline  322  13 

Clinton  1933  12 

Bedford  318  12 

Georgetown  558  12 

Quabbin  1359  12 
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ST8 


DIstrfct  Per-Pupll  Education  Reform  Aid      COiMBiNED  OVERSCORE  STB  98-00 


Harvard 

287 

26 

Tyngsborough 

1495 

26 

Westborough 

309 

25 

Stondiain 

319 

24 

North  Reading 

529 

23 

Hadley 

363 

18 

Sandwich 

326 

18 

M6dway 

1074 

17 

Amherst  Pelham 

1120 

16 

Northampton 

362 

16 

Bedford 

318 

16 

Central  Bert(Shire 

976 

16 

Nantucket 

306 

16 

Beriin  Boylston 

284 

16 

Brookline 

322 

15 

Quabbm 

1359 

15 

Oelchertown 

1641 

15 

Matconomet 

295 

13 

East  Longmeadow 

384 

13 

West  Bridge%vater 

344 

13 

Lawrence 

3643 

13 

Belmont 

318 

12 

Norwood 

342 

12 

Pentucket  Regional 

1057 

11 

Lenox 

301 

11 

Concord 

299 

11 

Douglas 

1981 

11 

Beverly 

385 

11 

Gardner 

2016 

10 

Lunenburg 

467 

10 

72 


TOTAL  GR8 


District  Per-Pupil  Education  Reform  Aid         GR8  TOTAL  iMCAS  98-00 


Harvard 

287 

71 

Tvnasboreuah 

1495 

60 

Stonehflin 

319 

58 

North  Rsadina 

52 

309 

fil 

Sflffuiwich 

326 

Hadley 

363 

45 

CMitral  Berkshire 

976 

43 

Mtedway 

1074 

42 

3643 

42 

Brooklinc 

322 

41 

Frontier 

473 

40 

Bedford 

318 

40 

1120 

40 

WW s^i  njuvwwwiiiid 

344 

39 

Noftfuffnoten 

362 

37 

Qtiabbin 

1359 

37 

Chelsea 

3318 

33 

385 

32 

»  fTi  11111  ^#WTt3H^II 

284 

31 

M 

WW 

wV 

Belchertown 

1641 

30 

Norwood 

342 

28 

East  Longmeadow 

384 

28 

Tantasqua 

1396 

27 

Ludlow 

1304 

26 

Westfbrd 

605 

26 

Berkshire  Hills 

330 

25 

Gardner 

2016 

25 

73 


ELA10 


DISTRICT  Per-Pupli  Education  Reform  Aid    COMBINED  OVERSCORE  ELA 10  98-00 


Stoneham 

319 

24 

Hatfield 

414 

23 

Harvard 

287 

20 

Shrewsbury 

552 

17 

Norton 

1558 

17 

Lenox 

301 

16 

Chelsea 

3318 

16 

Brainfree 

336 

14 

Nauset 

1021 

13 

Grafton 

807 

13 

Southbridge 

1782 

12 

WestMfood 

325 

12 

Belmont 

318 

12 

Hadley 

363 

11 

Ayer 

233 

10 

Littleton 

355 

10 

WestlxMough 

309 

10 

Millbury 

853 

10 

Needham 

315 

10 

Lee 

353 

9 

Fitehburo 

2269 

9 

IMarblehead 

333 

9 

Clinton 

1933 

9 

Berlin  Boylston 

284 

8 

Norwell 

321 

8 

Pentucket  Regional 

1057 

8 

Wobum 

344 

8 

Lawrence 

3643 

8 

Gill  Montague 

1184 

8 

Medway 

1074 

7 
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DISTRICT  Per-Pupll  Education  Reform  Aid  COMBINED  OVERSCORE  M10  98-00 


HAivsrd 

287 

42 

Lenox 

301 

34 

Hadley 

363 

30 

Lee 

353 

28 

Nauset 

1021 

28 

Shrewsbury 

552 

27 

Westborough 

309 

27 

Norton 

1558 

24 

Grafton 

807 

23 

Belmont 

318 

21 

Ayer 

233 

18 

North  bore  South  boro 

319 

18 

Waohusett  Reg. 

1797 

18 

Acton  Boxborough 

286 

16 

Georgetown 

558 

16 

Chelsea 

3318 

16 

Gouthbridge 

1782 

16 

Ware 

1732 

15 

Sharon 

295 

15 

Ipswich 

322 

15 

Mlllburv 

853 

14 

Westwood 

325 

14 

Pentucket  Regional 

1057 

14 

Old  Rochester 

286 

13 

Stoneham 

319 

13 

LitUeton 

355 

13 

Northampton 

362 

12 

Greenfield 

1136 

12 

Braintree 

336 

12 

Tyngsborough 

1495 

12 

STIO 


DISTRICT 

Per-Pupll  Education  Reform  Aid 

COMBINED  OVERSI 

Wesiborough 

309 

18 

Lenox 

301 

17 

Haivard 

287 

17 

Shrewsbury 

S52 

16 

Millbury 

853 

16 

Hadley 

363 

14 

Stoneham 

319 

13 

Chelsea 

3318 

12 

HatfieM 

414 

12 

Grafton 

807 

12 

Nauset 

1021 

12 

Belmont 

318 

11 

Norton 

1558 

11 

Westwood 

325 

11 

Berlin  Boylston 

284 

9 

Webster 

1018 

9 

North  Adams 

1947 

9 

Clinton 

1933 

8 

Pentucket  Region 

1057 

8 

Hudson 

425 

8 

Lee 

353 

8 

Northbridge 

2070 

8 

North  Reading 

529 

8 

Quaboag  Regions 

1627 

ft 

w 

Belchertown 

1641 

Medfield 

332 

Douglas 

1981 

Northboro  Southb 

319 

Nantucket 

306 

Wobum 

344 

7« 


TOTAL  GR10 


DISTRICT  Per-Pupll  Education  Reform  Aid         GR10  TOTAUMCAS  98-00 


Harvard 

287 

91 

Lenox 

301 

84 

Shrewsbury 

552 

76 

Westfoorough 

309 

66 

Hadley 

363 

64 

Nauset 

1021 

64 

Norton 

1558 

63 

Stoneham 

319 

58 

Hatfield 

414 

56 

Milibury 

853 

56 

Belmont 

318 

55 

Chelsea 

3318 

55 

Lee 

353 

53 

Grafton 

807 

51 

Westwood 

325 

46 

Braintree 

336 

41 

Pentucfcet  Regional 

1057 

40 

Southbridge 

1782 

39 

LitUeton 

355 

38 

Ayer 

233 

35 

Wachusett  Reg. 

1797 

34 

Northboro  Southboro 

319 

33 

Sharon 

295 

32 

North  Adams 

1947 

32 

Berlin  Boylston 

284 

32 

West  Boytston 

473 

30 

Belchertown 

1641 

28 

Brookline 

322 

28 

Need  ham 

315 

28 

Medway 

1074 

28 

77 


APPENDIX  D 

SCHCX)LDISTRICTS  THAT  MOST  OVER-  PERFORMED 
THEIR  DEMOGRAPHY  ON  TOTAL  2000  MCAS  SCORES 


The  following  tables  list  the  school  districts  that  most  over-performed  their 
demographic  characteristics  on  the  2000  MCAS  in  total  MCAS  score  (adding 
scaled  scores  in  English  Language  Arts,  Math,  and  Science  and  Technology  for 
Grades  4,  8,  and  10).  The  lists  represent  districts  that  most  over-performed 
based  on  Demographic  Rank  compared  to  Total  MCAS  Rank. 

For  example,  Braintree  is  ranked  #91  in  terms  of  where  it  fits  in  the  overall 
demographic  pattern  of  Massachusetts.  Its  overall  MCAS  score  (ELA,  Math, 
Science  and  Technology  for  Grades  4,  8,  and  10)  is  the  32nd  best  in  the  state. 
Thus,  Braintree's  total  MCAS  scores  are  59  ranks  above  the  performance 
predicted  by  its  demographic  characteristics. 

Key: 

TOTALSS  MCAS 2000  is  the  sumof  the  district's  sooresonthe  2000  MCAS.  This  indudes  Grade 
4,  8,  and  10  in  English  Language  Arts,  Math,  and  Sdence  aikl  Technology.  It  does  not  include 
Grade  8  History. 

DEMRANK  is  Demographic  Rank  which  is  derived  from  the  Effectiveness  Index  methodology 
which  sorts  communities  based  on  their  demographic  characteristics  (see  Appendix  F). 
Demographic  rank  sorts  districts  based  in  these  characteristics. 

SSRANK  is  Scaled  Score  Rank  which  is  where  the  district's  combined  MCAS  scores  rank 
compared  to  the  combined  MCAS  scores  of  the  rest  of  the  state's  districts. 

OVER-RANK  is  the  difference  betwe«i  where  the  district  is  ranked  based  on  its  demographic 
characteristics  (DEMRANK)  and  where  the  district  is  ranked  based  cm  actual  combined  MCAS 
scores  (TOTAL  SS  MCAS  2000)  relative  to  the  rest  of  the  districts  in  the  Commonwealth 
(DEMRANK-  SSRANK  is  the  formula).  The  higher  this  number,  the  more  a  district  over- 
performed  its  demographic  characteristics. 
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TOTAi      MPAQ  9nnn 

1 1  ML  OO  mv^MO  aUUU 

ncMRANK 

Braintree 

2168 

Q4 

yi 

oo 
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Douglas 

2118 

AAA 

144 

A4 

91 
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53 
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1  #1 
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1  ly 

CO 

02 

Stoneham 

2147 

4  no 
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60 
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4o 
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1 1  o 

DO 

4o 
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OO 
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40 
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99 

55 

44 
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43 
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OO 
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43 

Nortnampton 

2117 

4  0Q 

no 

40 

43 

Tyngsborough 

2157 

92 

49 

43 

Harvard 

2219 

40 

1 

39 

Brookiirie 

2180 

61 

23 

OO 

38 

MiMDury 

2123 

OO 

of 

Westborough 

2202 

47 

10 

o^ 

37 

Lenox 

2159 

77 

^O 

43 

O  A 

34 

MiHbrd 

2117 

127 

94 

33 

LXXDey  Cctanton 

2116 

yo 

OO 

32 

uxDnage 

04  OO 

2123 

4  10 

Of 

OO 

32 

Carver 

04  40 

2112 

4  Oil 
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4  r\A 
lU4 

OA 

30 

Grafloo 

2151 

82 

52 

30 

Hottirook 

2089 

170 

•4  JO 

142 

28 

Peabody 

2092 

166 

139 

27 

Southbnoge 

2069 

192 

166 

26 

Beverly 

2135 

95 

70 

25 

BiacKstone  MilMlle 

2111 

133 

4  no 

108 

25 

Hufl 

2097 

157 

132 

25 

^4orwoocl 

2141 

90 

65 

25 

Gardner 

2078 

180 

156 

24 

MettNjen 

2084 

173 

149 

24 

West  Bridgewater 

2121 

112 

88 

24 

East  Longmeadow 

2147 

81 

58 

OO 

23 

Ludlow 

2072 

184 

161 

23 

Stougnton 

2110 

136 

113 

23 

Leominster 

2078 

179 

157 

22 

Somerset 

2111 

132 

110 

22 

West  Boytston 

2164 

59 

37 

22 

Gloucester 

2096 

154 

134 

20 

Swansea 

2105 

140 

120 

20 

Beknont 

2209 

22 

3 

19 

hkxton 

2136 

88 

69 

19 

Quincy 

2091 

160 

141 

19 

Rockport 

2157 

67 

48 

19 

Betchertown 

2123 

102 

84 

18 

Dartmouth 

2094 

155 

137 

18 

Abington 

2114 

120 

103 

17 

North  Reading 

2177 

43 

26 

17 

Georgetown 

2163 

54 

38 

16 

Dennis  Yarmouth 

2100 

142 

127 

15 

Fairttaven 

2078 

169 

155 

14 

Ware 

2066 

183 

169 

14 

Lee 

2093 

151 

138 

13 

Greenfield 

2070 

176 

164 

12 

Littleton 

2166 

45 

33 

12 

Monson 

2112 

118 

106 

12 

Northtxidge 

2097 

145 

133 

12 

Sandwich 

2160 

53 

41 

12 

SouthwidcToNand 

2115 

114 

102 

12 

Spencer  East  Brookfield 

2098 

143 

131 

12 

AmestKiry 

2116 

107 

96 

11 

Everett 

2061 

185 

174 

11 

Roddand 

2099 

139 

128 

11 

APPENDIX  E 


DEMOGRAPHICALLY-  CHALLENGED  SCHCX)LDISTRICTS  THAT 
Over- PERFORMED  ON  THE  1998, 1999,  AND  2000  MCAS 


The  following  tables  list  demographically-challenged  over-performing  school 
districts  in  English  Language  Arts,  Math,  and  Science  and  Technology  for 
Grades  4,  8,  and  10.  Over-performing  districts  score  higher  on  MCAS  than 
would  be  predicted  by  their  demographic  characteristics.  The  following  lists 
represent  demographically-challenged  districts  that  had  the  highest  combined 
MCAS  over-performance  for  1998, 1999,  and  2000. 

Key: 

NOTE:  This  listing  only  indudes  the  50  communities  that  are  at  the  lower  end  of  the  state's 
demography.  Challenged  Massadiusetts.  Districts  that  do  not  fit  this  definition  are  not 
evaluated  in  this  App>endix. 

COMB  OVERSCORE  CHAL  xxxx  98-00  is  the  Combined  Overscore  on  all  three  MCAS 
administrations  for  the  relevant  Grade/ Subject  tests. 

(For  example,  ELA4  refers  to  the  Grade  4  English  Language  Arts  Tests  for  1998, 1999,  and  2000.) 
COMB  OVERSCORE  CHAL  xxxx  98-00  is  the  difference  between  the  demographically- 
predicted  score  and  the  actual  district  MCAS  scores  for  all  tests,  all  administrations.  The 
higher  the  Combined  Overscore,  the  more  a  district  exceeded  its  demographic  characteristics. 
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DISTRICT       COMB  OVERSCORE  CHAL  ELA4  98-00 

LawrMKM  10 

Ayer  9 

Everett  9 

Oranfle  8 

Sprinofield  4  ELA  GR4 

Falrtiaven  4 

Clinton  3 

Holbrook  3 

Chelsea  3 


DISTRICT         COMB  OVERSCORE  CHAL  M4  98-00 

Chelsea  18 

Orange  16 

Ayer  15 

Everett  15 

Clinton  14 

Methuen  11  MATH  GR4 

Holbrook  8 

Lawrence  8 

Woroester  6 

Darfanoutti  4 

Springfield  4 

Pittsfield  4 


DISTRICT         COMB  OVERSCORE  CHAL  ST4  98-00 

Orange  14 

Everett  13 

Ayer  12  SCIENCE  & 

Chelsea  8  TECHNOLOGY  GR4 

Lawrence  6 

Berlcshire  Hills  6 

Fail  River  5 

Worcester  4 

Pittsfield  4 

Dartmouth  3 


DISTRICT 

COMB  OVERSCORE  CHAL  T0T4  98-00 

Orange 

38 

Everett 

37 

Ayer 

36 

Chelsea 

29 

TOTAL 

Lawrence 

24 

GR4  MCAS 

Clinton 

17 

Worcester 

12 

Springfield 

10 

Dartmoutii 

8 

Mettiuen 

9 

Holbrook 

9 

DISTRICT  COMB  OVERSCORE  CHAL  ELA8  98-00 

Lawrence  13 

Clinton  10 

Berkshire  Hills  9 

Gardner  7 

Chelsea  7 

Ludlow  7  ELA  GR8 

Southbridge  7 

Hull  6 

Quincy  6 

Leominster  5 

Dartmouth  4 

Methuen  3 

Southern  Berkshire  3 

District  COMB  OVERSCORE  CHAL  M8  98-00 

Chelsea  17 

Lawrence  16 

Ludlow  14 

Clinton  12 

Southern  Berkshire  11  MATH  GR8 

Acushnet  10 

Peabody  10 

Ware  9 

Greenfield  8 

Methuen  8 

Gardner  7 

Berkshire  Hills  6 

District  COMB  OVERSCORE  ChlAL  ST8  98-00 

Lawrence  13 

Gardner  10 

Berkshire  Hills  10 

Peabody  9 

Chelsea  8 

ainton  8  SCIENCE  & 

Everett  6  TECHNOLOGY  GR8 

Ludlow  6 

Southem  Berkshire  4 

Leominster  4 

Southbrklge  4 

North  Adams  3 

Chioopee  3 

New  Bedford  3 

Dartmouth  3 

Quincy  3 


District  COMB  OVERSCORE  TOT  CHAL  GR8  98-00 

Lawrenoe  42 

Chelsea  33 

ainton  30 

Ludlow  26 

Berkshire  Hills  26 

Gardner  25  TOTAL 

Peabody  21  GR8  MCAS 

Southern  Berkshire  18 

Southbridge  14 

Leominster  14 

Acushnet  12 

Quincy  11 

Ware  9 

Methuen  9 


DISTRICT       COMB  OVERSCORE  CHAL  ELAIO  98-00 

Ch«isea  16 

Sotrthbridge  12 

Ayer  10 

Fitohburg  9 

Clinton  9  ELA  GR10 

Lawrence  8 

Hull  6 

Lowell  5 

Dartmouth  4 

Ware  4 

Wareham  3 

DISTRICT         COMB  OVERSCORE  CHAL  Ml 0  98-00 

Ayer  18 

Chelsea  16 

Southbridge  16 

Ware  15 

Greenfield  12 

Lowell  11 

North  Adams  10 

Gardner  10  MATH  GR10 

Ludlow  10 

New  Bedford  8 

Lawrence  7 

.  Fitchburg  5 

Palmer  5 

Provincetown  S 

Webster  4 

Ber1(shire  Hills  3 

Dartmouth  3 

Revere  3 

DISTRICT        COMB  OVERSCORE  CHAL  ST10  98-00 

Chelsea  12 

Webster  9 

North  Adams  9 

Clinton  8 

Lowell  5  SCIENCE  & 

Lawrence  6  TECHNOLOGY  G  RIO 

Quincy  4 

Southbridge  4 

Fitchburg  4 

New  Bedford  4 

Hull  3 

Ware  3 


DISTRICT  COMB  OVERSCORE  TOTAL  98-00 

Chelsea  65 

Southbrkloe  39 

Ayer  35 

North  Adams  32 

FHchbury  27  TOTAL 

Clinton  26  GR10  MCAS 

Lowell  25 

Ware  25 

Lawrence  23 

Provincetown  21 

Webster  18 

New  Bedford  15 

Palmer  9 


APPENDIX  F 
DERIVING  THE  EFFECTIVENESS  INDEX 


Methodology  of  the  Effectiveness  Index 

The  Effectiveness  Index  (EI)  is  derived  by  comparing  actual  scores  on 
standardized  tests  with  scores  as  predicted  by  a  model  which  factors  in  the  role 
community  characteristics  play  in  educational  outcomes. 

The  Community  Effects  Factor  (CEF)  model  was  developed  in  a  doctoral 
dissertation  (Education  Achievement  Communities:  A  New  Model  for  "Kind  of 
Community"  in  Massachusetts  Based  on  an  Analysis  oj Community  Characteristics 
Affecting  Educational  Outcomes,  May  1998,  University  of  Massachusetts,  Amherst). 
That  work  is  the  basis  for  determining  school  effectiveness.  The  model  examines 
the  relationship  between  selected  demographic  characteristics  and  educational 
outcomes.  These  characteristics  include:  average  education  level,  average 
income,  poverty  rate,  single-parent  status,  language  spoken,  and  percentage  of 
school-age  population  enrolled  in  private  schools.  These  variables  were  chosen 
because  they  correlate  with  achievement  and  because  the  education  literature 
identifies  them  as  connected  to  academic  performance. 

In  order  to  refine  a  better  CEF  model,  it  is  first  necessary  to  factor  the  impact  of 
these  demographic  variables  on  each  other.  This  can  be  done  through  a 
technique  known  as  principal  components  analysis  that  is  a  statistical 
mechanism  that  reduces  many  variables  to  a  few  salient  ones  that  have  the  most 
impact  on  an  outcome.  Once  the  factors  have  been  identified,  a  regression 
analysis  produces  the  equations  that  can  be  used  to  either  build  a  kind-of- 
commimity  model  or  to  predict  expected  district 

performance  on  achievement  tests.  The  degree  to  which  a  community's 
characteristics  lifts  or  lowers  test  scores  is  reflected  in  a  Community  Effects 
Factor  (CEF),  a  measure  of  demography. 

The  CEF,  which  is  a  measure  of  the  demographic  lift  or  drag  of  each  community 
concerning  educational  achievement,  is  a  good  point  of  departure  for  analyzing 
school  and  school  district  effectiveness.  The  CEF  identifies  expected  levels  of 
performance  based  on  community  characteristics  which,  for  better  or  worse,  are 
very  powerful  indicators  of  educational  achievement  in  Massachusetts.  In  this 
analysis,  Weston  is  the  most  demographically  advantaged  community  in  the 
state  in  terms  of  educational  outcomes  (CEF  =  +  Z8),  and  Lawrence  is  the  least 
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advantaged  (CEF  =  -  4.8).  The  CEF  has  a  strong  relationship,  or  correlation,  to 
test  scores. 

Correlation  is  a  process  that  identifies  the  interdependence  of  one  variable  with 
another.  Correlation  simply  shows  "the  extent  to  which  two  things  typically  run 
together."  [The  Economist,  6  Dec.  1997,  p.  82].  Correlation  is  not  equivalent  to 
causation;  it  can  only  reveal  tendencies  between  variables,  not  identify  causes. 
Correlations  simply  demonstrate  relationships.  A  perfect  correlation  would  be 
1.0.  For  example,  the  correlation  between  inches  and  feet  is  1.0  because  it  is  a 
perfect  linear  fit;  12  inches  always  equals  one  foot  Correlations  in  real  world 
situations  involving  htunan  behavior  are  never  1.0. 

The  correlation,  or  the  connection,  between  spending  (Per-Pupil  Expenditure  or 
PPE)  and  achievement  in  Massachusetts  is  relatively  low.  While  spending  dearly 
matters,  merely  increasing  spending  levels  has  a  relatively  weak  impact  on 
results.  Increasingly,  many  people  are  coming  to  the  realization  that  how  a 
system  spends  money  is  more  important  than  how  much  money  it  spends.  The 
achievement  outcome  accounted  for  by  the  community  effects  factor  (CEF)  is 
much  stronger;  that  relationship  is  .83.  This  is  not  to  say  the  community  context, 
the  CEF,  is  the  most  important  determinant  of  school  success,  but  it  is  a 
significant  element  that  must  be  a  major  consideration  in  any  plan  to  improve 
education  in  disadvantaged  areas. 

The  Effectiveness  Index  (EI)  is  generated  in  the  following  manner: 

•  Utilize  the  1998  MCAS  results  as  an  outcome  indicator  for  achievement  in 
each  of  the  state's  most  populous  240  or  so  districts  (the  exact  number  depends 
on  grade  level).  (NOTE:  This  model  does  not  evaluate  results  in  the  smallest 
communities  of  the  state  which  comprise  about  2%  of  the  population.) 

•  Utilize  the  EI  model  to  predict  a  score  for  each  district  This 
predicted  score  is  based  solely  on  community  characteristic  as  they  affect 
educational  outcomes. 
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•    Compare  the  actual  to  the  predicted  score.  Systems  whose  actual  scores 
are  significantly  higher  than  predicted  scores  and  whose  absolute  scores  are  at  or 
above  state  average  are  identified  as  Effective.  Systems  with  positive 
effectiveness  indexes  but  scores  below  state  average  are  identified  as 
Noteworthy. 

The  basis  for  this  model  was  developed  as  a  doctoral  dissertation.  The  following 
provides  detail  on  the  statistics  behind  the  model  which  is  used  to  predict 
expected  scores  based  on  demographics. 

Robert  D.  Gaudet 
rgaudet@world.std.com 
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The  Statistical  Model 


Response:  TOTSSOO 


Summary  of  Fit 

RSquare 

0.83115 

RSquare  Adj 

0.825954 

Root  Mean  Square  Error 

22.31567 

Mean  of  Response 

2114.183 

Observations  (or  Sum  Wgts) 

202 

Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>ltl 

Lower  95% 

Upper  95% 

Std  Beta 

Intercept 

2106.8307 

4.026321 

523.26 

<0001 

2098.8899 

2114.7715 

0 

F22C0L 

38.86942 

7.040252 

5.52 

<.0001 

24.984448 

52.754392 

0^72729 

F2Z  FAM$ 

37.955354 

6.562543 

5.78 

"tOOOl 

25.012531 

50.8981 76 

0364934 

F2ZP0V 

-47.30046 

9.446159 

-5.01 

<.0001 

-65.93042 

-28.67049 

-0.30435 

F2ZPRVSCHL 

-11.58736 

3.795885 

-3.05 

0.0026 

-19.07371 

-4.101016 

-0.09911 

F2ZN0NENG 

-8.4744 

2.70125 

-3.14 

0.0020 

-13.80188 

-3.146922 

-0.14607 

F2ZSPAR 

1.0757288 

6.715529 

0.16 

0.8729 

-12.16882 

14.320273 

0.005472 

(Effect  Test 


Source 

Nparm    OF    Sum  of  Squares 

F  Ratio 

Prob>F 

F2ZC0L 

1       1  15179.572 

30.4817 

<.0001 

F2Z  FAM$ 

1       1  16657.949 

33.4504 

<.0001 

F22  POV 

1       1  12486.477 

25.0738 

<.0001 

F2ZPRVSCHL 

1       1  4640.479 

9.3184 

0.0026 

F2ZN0NENG 

1       1  4901.263 

9.8421 

0.0020 

F2ZSPAR 

1       1  12.778 

0.0257 

0.8729 

Whole-Model  Test 
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1950  2000  2050  2100  2150  2200  2250 
TOTSSOO  Predicted 


Analysis 

Of 

Variance  ) 

Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

6 

478004.36 

79667.4 

159.9782 

Error 

195 

97107.86 

49ao 

Prob>F 

C  Total 

201 

575112.22 

<J000^ 
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